Bounding an Autonomous Agent: Tool Calling, Episodic Memory, and the Failure Modes of Unbounded Loops

The Context: "The Agent That Would Not Stop"

Khepri is my autonomous coding agent: it reads a repository, decides which files matter, runs tools, and proposes changes. By September it was genuinely useful for mechanical work — renaming a symbol across forty files, tracing a config value through the tree, generating a migration.

In October it ruined an afternoon by trying to delete 300MB of node_modules because a task said "clean up build artifacts" and it never stopped trying. It retried the same deletion eleven times, escalating from polite requests to permission prompts, before I killed it.

Nothing about that behaviour was a language-model failure. The model made a reasonable decision eleven times in a row given the state it was shown. The failure was entirely architectural: I built a loop with no termination condition, no budget, and no notion that "it did not work" is a result.

The Problem Statement

Make an agent that can take real actions on a real machine without requiring a human in the loop for every step:

Termination: Guaranteed halt under every condition, including partial failure and infinite retry.
Budget: Hard ceilings on tool calls, wall time, and destructive operations.
Memory: Recalls prior sessions without unbounded growth or unbounded retrieval.
Authority: Never escalates privilege on its own, and never repeats a refused action.
Explainability: Every action is traceable to the observation that motivated it.

Visual Architecture

The Core Logic / Process

1. The Loop Has a Budget, Not a Suggestion

Every agent loop I consider shipping now carries the same triple guard, and the budget check happens before the tool call, never after:

interface Budget {
  maxToolCalls: number;
  maxWallClockMs: number;
  maxDestructiveOps: number;
}

function shouldHalt(
  b: Budget,
  used: Budget,
  noProgressStreak: number,
): string | null {
  if (used.maxToolCalls >= b.maxToolCalls) return "tool_call_budget_exhausted";
  if (Date.now() - start >= b.maxWallClockMs)
    return "wall_clock_budget_exhausted";
  if (used.maxDestructiveOps >= b.maxDestructiveOps)
    return "destructive_budget_exhausted";
  if (noProgressStreak >= 3) return "no_progress_detected";
  return null;
}

The noProgressStreak term is the one that would have saved my afternoon. If three consecutive iterations produce no new information — the same error, the same file listing, the same empty result — the agent stops regardless of remaining budget. Eleven retries of an identical destructive deletion produce identical observations, so the streak hits three and the loop halts with a clear reason.

Without it, budget is the only backstop, and budget large enough for legitimate hard tasks is always large enough to permit an infinite failure loop too.

2. Refusals Are Permanently Recorded

The single most important line of state in the whole agent is the set of refused actions:

// Once denied, never re-request — not after a retry, not after
// context compaction, not in a new session.
if (denied.has(actionKey)) {
  return { halt: `action previously refused by user: ${actionKey}` };
}

An agent that re-proposes an action the user explicitly declined is not being persistent, it is being disobedient. Human denial is not a transient obstacle to route around; it is a decision. Recording it durably — in the same store as memory, so it survives summarisation — is what separates a tool-using agent from an automated spam cannon pointed at itself.

The node_modules incident had exactly this shape: the first deletion was refused by the filesystem, the agent retried with variations, and nothing in its state recorded that the category was off-limits.

3. Policy Gates Destructive Work Separately

Tool calls are classified before execution, and anything irreversible or outside the workspace requires approval:

const DESTRUCTIVE = new Set([
  "fs.rm",
  "fs.rmdir",
  "shell.exec",
  "git.push",
  "fs.write.outsideRepo",
]);

async function authorize(call: ToolCall): Promise<Decision> {
  if (!DESTRUCTIVE.has(call.name)) return { allow: true };
  if (!withinScope(call))
    return { allow: false, reason: "outside_declared_scope" };
  return { allow: await requestHumanApproval(call) };
}

Two gates, in order, because they answer different questions. Scope is a machine decision — did the user ask for this, is the path inside the repository. Approval is a human decision — do you consent to this irreversible thing. Running scope before approval means the human is never asked about actions that were never going to be permitted anyway, which keeps the interruption rate low enough that people actually read the prompts.

4. Episodic Memory That Shrinks

Storing full transcripts is the obvious design and it fails quietly: retrieval quality degrades as the corpus grows, because the most relevant episode competes with two hundred irrelevant ones.

I store summarised episodes with explicit retrieval, not raw dumps:

interface Episode {
  id: string;
  goal: string; // what was attempted
  outcome: "success" | "failure" | "refused";
  lessons: string[]; // <= 5 distilled facts
  toolFingerprint: string; // which tools/shapes this resembled
}

// Prune: keep all failures (rare, high value), cap successes per shape.
episodes = prune(episodes, { maxSuccessPerShape: 3, keepAllFailures: true });

Failures are kept forever because they are rare and disproportionately informative. Successes are capped per fingerprint because discovering "I already searched for this symbol" three times is useful and the fourth time is noise. The store is therefore bounded and biased toward what the agent most needs to remember.

The "Baigan" Take (Expert Insights)

The reflexive worry about AI agents is hallucination — the model confidently stating something false. In my experience the operational danger is the opposite: the model is confidently trying, and the system around it has no notion of enough.

baigan ka agent is the belief that "if the model is smart enough, the loop will terminate itself." It will not. A capable model confronting a persistent environmental error will generate an unbounded number of increasingly creative attempts, because from its perspective every retry is a rational response to new information. Eleven retries was not stupidity; it was faithfulness to a loop with no exit.

Four rules I now hold to:

Every loop needs a halt path that is not budget exhaustion. Budgets are a backstop. The noProgressStreak check is the real termination condition, because it detects the failure mode you actually hit rather than the one you can afford to hit.
Human refusal is a permanent state, not an error to retry. This costs nothing to implement and it is the difference between an assistant and an adversary. Store refusals outside the context window so compaction cannot erase them.
Separate "may I" from "should I". Machine-checkable scope first, human approval second. Merging them into one prompt means asking people about things that were never permitted, training them to click through.
Cap successes, keep failures. An agent that has solved the same class of task four times does not need a fifth memory of it; an agent that failed once absolutely needs that memory. Memory without a pruning policy becomes noise with a token cost.

The generalisable principle, which applies well beyond agents: any system that acts on the world needs an explicit account of when to stop, what it may not repeat, and what it must remember. Without those three, autonomy is not capability — it is an unbounded loop with good PR.

Configuration