The agent loop: feeding results back to the model and knowing when to stop

The loop is yours, not the model's

A call to a language model produces one response and ends. The model cannot run a tool, wait for the result, and keep thinking on its own: when it asks for a tool, its turn ends right there. If someone asks “how much has my bench press improved this month?”, the model can first ask for the workout history, but it will not see that history until someone sends it in a new call.

That “someone” is the harness: the program around the model. It receives the request, runs the tool, adds the result to the conversation, and calls the model again. It repeats the process until the model answers without asking for anything else. What we usually call an “agent” is, at its core, that loop written in the application's code. The model decides what to ask for; the loop decides whether it runs, how many times it repeats, and when it stops.

The full cycle

Each pass through the loop is a round: one model call, running the tools it asked for, and sending their results back. The loop has two exits. The normal one happens when the model answers without asking for tools: that is the final answer for the user. The safety one happens when the maximum number of rounds runs out, even if the model keeps asking for tools.

The loop, step by step

This is the loop in pseudocode. The language and the API do not matter: what has to be done is the same in every case. Notice three things: when the loop exits, what is added to the history on each pass, and when the model is called again.

turn = call_model(history)

repeat at most MAX_ROUNDS times:
    if the turn asks for no tools:
        exit                           # normal exit: this is the final answer

    results = []
    for each tool request in the turn:
        result = run_tool(request.name, request.arguments)
        add (request.id, result) to results

    append the turn to the history     # what the model asked for
    append the results to the history  # what the app answered
    turn = call_model(history)

return turn

That is the core. “Repeat at most” enforces the round limit, and “exit” is the normal exit. On each pass the tools run one at a time, in the order the model asked for them. That is slower than running them in parallel, but it keeps two writes to the same data from racing each other. Changing that decision means changing the loop, not the model. The only thing missing is what to do when the loop ends because the rounds ran out, and that has its own section below.

What gets fed back, and in which order

Gymnasia keeps the chat on the phone. Before calling any provider, one local rule selects up to the latest 20 messages in the thread. Each adapter converts that selection to its API's format. Then, within that request, each round adds the model's turn and the tool results. The entire active sequence is sent again in subsequent rounds: it is not trimmed halfway through a chain of requests and results.

Order matters. The model's request comes first, followed by the result with the matching identifier. Anthropic, for example, rejects a tool_result that does not follow the turn containing its tool_use. Context selection follows one policy; the API changes how that sequence is represented:

The 20 messages are the upper bound of the initial selection. Google may send fewer: its adapter removes older exchanges when the request exceeds its size limits, while keeping the active turn's steps intact. That adjustment happens after context selection.

Previously, Gymnasia used previous_response_id: OpenAI kept the response, so sending only new results was enough. Now every OpenAI call carries store: false, and Gymnasia replays the chosen context and active sequence; it no longer relies on OpenAI storing a response to continue it. Encrypted reasoning stays in memory only during the request and is neither shown nor saved in the chat. This makes it possible to decide later which context to summarize or exclude, although Gymnasia does not compact it yet.

store: false prevents responses from being stored as application state at OpenAI; it is not a promise of zero retention for other purposes, such as abuse monitoring.

Why you need a limit

Nothing forces the model to stop asking for tools. It can repeat the same call because it is not happy with the result, chain searches without reaching a conclusion, or react to an error by asking for the tool again. Each round is a network call, with its latency and token cost, and the history grows on every pass. Without a limit, a loop like that can leave the user waiting indefinitely.

Gymnasia sets the limit in a constant shared by the three providers: MAX_TOOL_ROUNDS = 10. The number is a product decision, not a technical truth. It has to be high enough for real queries that chain several tools, such as reading the history, computing a personal record, and saving a goal, and low enough to cut a runaway loop before the wait or the bill becomes noticeable.

What happens when the rounds run out

Stopping the loop is the easy part. The hard part is deciding what to tell the user. The first version of Gymnasia got this wrong, and it is worth describing because it is easy to repeat in any harness:

• Google threw a technical error: the response was still waiting for tools when the limit was reached.
• OpenAI and Anthropic left the loop silently and returned the last turn, which still asked for tools. Because the text the model writes between rounds (“let me look up your history…”) accumulates, the user saw that half-finished sentence as if it were the final answer, or a generic “the provider returned no content” error.

Either way, the user got no useful answer and no hint that the round budget had run out. Stopping abruptly throws away the work already done: the agent may already have read the history and been one step away.

The fix: one last call in which the model can only write

When the rounds run out, the loop does not stop dead. It makes one last call to the model in which it cannot ask for any tool, neither the pending ones nor any other: it can only answer with text, using what it has already found out.

First there is one detail to settle. The model's last turn still contains tool requests, and providers require every request to have a result. So the loop answers them without running them, with a result that only says what happened. In one closing API call, Gymnasia sends those results in the history and puts the direction on how to reply in the system instructions:

# The rounds ran out and the last turn still asks for tools.
if the turn asks for tools:
    for each pending request:
        result = "Not executed: the step limit for this answer was reached."
    append the turn and those results to the history

    instructions = system_prompt + "You have reached the step limit for this answer.
        Answer with what you already know, explain the limit,
        say what was left undone and ask whether to continue."
    turn = call_model(history, instructions, tools = none)

return turn

These are two parts of the same API request: the history says “this tool was not executed”, and the instructions say “explain the limit and ask whether to continue”. With tool_choice: "none", the model cannot ask for another tool. So the user gets something like “I found your history, but I reached the step limit for this answer and did not compute the record. Do you want me to continue?”. If they say yes, the agent carries on: the limit is counted per message, so the new message starts with ten fresh rounds. And if the pending tool was a write, such as saving a goal, the model knows it was not done and will not claim otherwise.

Why the instruction does not go in the tool result

The first version put everything in the result: “Not executed… answer with what you know and ask whether to continue”. It worked with Google, but not with OpenAI. The model treated that text as data returned by a tool, not as an instruction: it did not ask, and sometimes it even said it had no access to the tools.

That makes sense. A tool result can contain text you do not control, such as a web page or a document, and a model that obeyed instructions written there would be easy to manipulate. After moving the instruction to the system instructions of the closing call, all three tests with OpenAI explained the limit and ended by asking whether to continue.

How the model is told not to use tools

All these APIs have a parameter, tool_choice, that decides what the model may do with tools in a given call. With auto, the usual value, the model decides whether to ask for one. With none, the tools stay declared, but the model cannot ask for any and has to answer with text. The closing call uses none. Each provider writes it differently, and there are two traps: