Loop engineering: the loop is the product

An agent is a model in a while loop. That is the entire architecture, and it is worth saying plainly because the industry has spent two years optimising the wrong term. Teams rewrite the system prompt for a fortnight and never write down what the loop does when a tool times out, how many times it may try again, what it carries forward between turns, or what condition ends it.

We call the discipline of specifying those things loop engineering, and in production it is where the wins are. The prompt determines how well one step goes. The loop determines whether a hundred steps compound into an answer or into a bill.

Four things a loop must decide

Every agent loop, however it is dressed up, makes the same four decisions on every pass. Write them down before you write the prompt.

1. What goes in. Not the whole history — the specific slice of state this turn needs. This is a decision, and defaulting to append-everything is still a decision, just an expensive one.
2. What may act. The tools available on this turn. Not the whole registry: the subset that makes sense given where the loop currently is.
3. What comes back. How a tool result is validated and reduced before it re-enters context. Raw tool output is the single largest source of context rot.
4. Whether to continue. The stopping condition, stated as something checkable rather than as a hope that the model will say DONE.

Context is a budget, not a bucket

The default loop appends. Turn one is small, turn twenty is enormous, and somewhere around turn twelve quality falls off a cliff that nobody can point at in a trace, because nothing failed — the model simply had four thousand tokens of stale tool output between it and the thing that mattered.

Treat every token in context as something you chose to put there. If you cannot name why a block is still present on turn nine, it should not be.

In practice that means three moves. Summarise completed sub-tasks down to their result and drop the transcript that produced it. Keep a small structured scratchpad the loop owns — goal, findings so far, open questions — and rewrite it rather than appending to it. And keep large artefacts out of context entirely: write them to a store, and pass a handle.

// The shape we reach for. Note what is explicit:
// budget, tool subset per phase, reduction on the way back in,
// and a stopping condition that is not "the model said so".

while (state.phase !== "done") {
  if (budget.spent > budget.cap) return bail(state, "budget");
  if (state.turn > MAX_TURNS)     return bail(state, "turns");

  const ctx   = compose(state);            // chosen, not accumulated
  const tools = toolsFor(state.phase);     // narrowed, not the full registry
  const step  = await model(ctx, tools);

  if (step.tool) {
    const raw = await invoke(step.tool);   // may throw, may time out
    state = reduce(state, validate(raw));  // shrink BEFORE it re-enters
  }

  state = advance(state, step);            // phase transition is code, not vibes
}

Retries are where the money goes

The failure we are called in to fix most often is not a wrong answer. It is a loop that retried a failing tool forty times at three in the morning, each attempt carrying the full accumulated context, and produced a five-hundred-dollar line item and no output.

Retries need three properties, and they are cheap to add on day one and painful to retrofit. Cap them per tool rather than globally, so a flaky search does not consume the whole budget. Back off exponentially with jitter, because synchronised retries turn one slow dependency into an outage. And make the retry visible to the model: a bare failure teaches it nothing, while "this call failed twice with a timeout, you have one attempt left" reliably produces a different strategy.

Phases beat freedom

An agent with fifteen tools available on every turn will use the wrong one eventually, and the more capable the model the more creatively wrong the choice. Splitting the loop into phases — gather, then analyse, then act — and exposing only each phase's tools removes an entire category of failure without constraining anything the task actually needed.

It also makes the loop debuggable. When a run goes wrong you can ask which phase it went wrong in, and the answer is a fact rather than an interpretation of a transcript.

What we check before anything ships

• Every loop has a hard turn cap and a hard cost cap, both enforced in code rather than in the prompt.
• Every tool has its own retry budget, its own timeout, and a defined behaviour on exhaustion.
• Tool output is validated and reduced before it enters context — never appended raw.
• The stopping condition is checkable by the loop itself, and there is a defined bail state for every way it can fail to be met.
• Every run leaves a trace showing phase transitions, tool calls, retries, and tokens spent. If you cannot reconstruct a run afterwards, you are not operating it, you are hoping.

None of this is exotic. It is the same discipline any long-running background job has needed for thirty years, applied to a component that happens to be probabilistic. The novelty is entirely in the model. The engineering around it is old, and it is the part that decides whether the thing survives contact with a real business.