Tool errors: how an agent recovers and when it must stop

Tool errors: how an agent recovers and when it must stop

The tool returned something, but what happened?

The user asks “How much did I weigh yesterday?” The model does not know the weight: it needs to request a lookup in the app’s records. That query might find 76 kg, establish that no weight was logged that day or fail because storage cannot be read.

These are three different situations. A retrieved value lets the agent answer. A missing record lets it explain what is absent and perhaps look for a reference. A storage failure prevents it from knowing what was there. A generic “Error” message does not communicate that distinction.

To understand where the failure occurs, start with the cycle. The model proposes a tool call, a request containing the function name and its arguments. It stops generating its answer while waiting for the result. Execution happens next, in the harness.

The tool cycle in four steps
  1. The model requests an operation.

    For example, looking up the weight for a date.

  2. The harness validates the request.

    It checks that the function exists and its arguments are allowed.

  3. The harness runs the function.

    It obtains the value or detects why the operation failed.

  4. The model receives the result.

    If continuation is allowed, it answers the user or proposes another tool call.

Returning the result does not run the function again. The model continues from what the harness just did. If the failure requires stopping the turn, the harness displays a notice and does not request another model response.

A result must say what happened and what is allowed next

An error contract is an agreement on how failures are communicated. It separates three questions: did the operation fail, why, and what can happen next? An error flag answers the first; a code and message explain the cause; a recovery rule limits continuation.

The program that ran the operation creates the flag. It does not derive it by searching for the word “error” in retrieved content. A note’s text belongs to the user’s data; the query status describes whether the app managed to read it.

Gymnasia uses three names for its recovery rules. You do not need to know them beforehand: each represents a concrete harness decision.

What happenedWhat is allowed nextName in the contract
The arguments do not have the accepted shape.The model can correct them. The operation has not run yet.correct_arguments
A query does not find the requested record.The model can propose another useful query within the user’s request. Every new call is validated.choose_alternative
The program decides it cannot safely continue.The harness stops the turn. The model is not allowed to keep calling tools.stop_turn

The following JSON shows the internal contract for a missing record. Notice that it allows another query, but the message only claims the value is missing for that date. The message remains in Spanish, as in the case study:

{
  "kind": "tool_error",
  "version": 1,
  "is_error": true,
  "error": "not_found",
  "message": "No hay registro para esa fecha. No implica que no haya registros en otras fechas.",
  "recovery": "choose_alternative",
  "recoverable": true
}

is_error identifies the failure, error lets the program recognize its cause without interpreting a sentence, and recovery specifies the permitted next step. recoverable: true means that continuation through that path is allowed, not that any action can be repeated. kind and version identify the format.

This JSON is an application decision. It is neither a mandatory output field nor a shared output schema accepted by all providers. The input schema, which defines the arguments a tool accepts, answers a different question: whether a request is valid before execution.

Answering when the weight from a month ago is missing

Coach helps users review their weight history. A natural question would be “How much did I weigh a month ago?” To turn that expression into a date, the harness must provide a reliable time reference. In the following example, we assume that step is resolved and there is no weight for the requested day.

User: How much did I weigh a month ago?

Coach: You have no weight logged for that day. The day before, you logged 76 kg.

This is an illustrative example. It shows how the agent could help: it receives a missing-record error and, if the harness allows continuation, looks up the previous day as a reference. The user does not need to write instructions for handling a failure.

The answer must distinguish the dates. The 76 kg does not answer the exact day requested: it belongs to the previous day. If no useful reference is found either, the agent says so. Looking up another date does not authorize inventing a weight or modifying records.

The real Gymnasia test checked a narrower sequence: querying a day with no weight and then another day explicitly specified in the test request. It verified recovery through a second read, but does not show that Coach independently searches for the nearest day for every relative-date question.

Sending the same status to three APIs

The harness decides what happened before preparing the next provider request. An adapter, the component that translates the result into each API’s format, then preserves that decision. It also preserves the call identifier, which associates the response with the operation the model requested.

Read the table from left to right: where the result goes, how the failure is communicated and which field connects it to the request.

API and resultHow the failure is sentLink to the call
OpenAI Responses
function_call_output
The error JSON is serialized inside output. No top-level is_error is added to that object.call_id
Anthropic Messages
tool_result
The block carries is_error: true and the error content in content.tool_use_id
Google Interactions
function_result
The result carries is_error: true and the error content in result.call_id

The format accepted by the API changes; the meaning of the result does not. The operation error originates in the app, and the provider delivers it to the model. The Google comparison refers to Interactions.

Catching the failure where the operation runs

Validation checks the arguments before calling the handler, the function that performs the work. For example, it rejects an incorrectly formatted date before querying storage. Once the request has been validated, errors can still occur while that function runs.

An expected absence, such as a day with no weight, is returned through the normal contract. An unexpected exception interrupts execution, for example when storage stops responding. The block that catches those exceptions must surround the local operation; catching only the request to OpenAI, Anthropic or Google would leave that failure outside it.

This pseudocode summarizes the behavior checked in Gymnasia. An unexpected exception during a save stops the turn directly. The next section explains a design option that allows verification before deciding to stop.

check the tool name and the accepted argument shape
if the request is invalid:
    return a marked error that allows argument correction

try to execute the operation
if it returns an expected error:
    apply its recovery rule
if an unexpected exception interrupts a read:
    return a safe, marked error
if an unexpected exception interrupts a save:
    stop the turn locally
    do not repeat the save or request another model response

if continuation is allowed:
    send the result and its status to the model

The difference lies in the operation’s effect. A failed read adds no data. An interrupted save may have changed the record before losing confirmation. That is why this implementation does not repeat it automatically.

Suppose the provider correctly delivered a request to log a meal, but the app’s storage failed. Calling it a “Provider error” would point to the wrong component: the problem appeared after the request was received, during local execution.

The developer may need the original exception to investigate. The model needs to know which operation could not be confirmed and which checks are permitted. The user needs a useful answer: the meal was saved, or the record needs checking because verification failed. File names and internal paths do not help them make that decision.

Checking a meal before saving it again

Logging a meal adds its calories, protein, carbohydrates and fat to the day’s totals. If a meal the user ate once is logged twice, those totals count two portions. The error can make it seem that the user has exceeded their macro targets.

Imagine the user asks “Log my lunch: 200 g of cooked rice and 150 g of chicken.” The app tries to save it, but gets no confirmation. It may have failed before adding it or after doing so. The error notice alone cannot distinguish those cases.

A harness can be designed to allow a read-only query before deciding that it must stop. The model requests a check of what was saved, without adding another meal. The query must identify this specific action: finding chicken and rice in the log is insufficient if a similar meal was already there.

The check has three possible outcomes:

It was saved

There is evidence that this meal was recorded. Coach can say “I’ve logged your meal.” It does not need to tell the user about the technical failure that has been resolved.

It was not saved

There is sufficient evidence that the action was not completed. The harness can permit finishing the pending save, following its rules and avoiding duplicates.

It is still unknown

The query fails or provides insufficient evidence. The turn stops with a notice: “I can’t confirm whether the meal was saved. Check your log before adding it again.”

This recovery is a design option, not the path taken in the Gymnasia test. In that test, a save exception stopped the turn without requesting another model query. If verification is introduced, it must happen before applying stop_turn. Once a stop has been ordered, the model cannot ignore it.

A capable model can choose a good check. The harness must provide the permissions and evidence that make it reliable. Simply telling the model “do not duplicate meals” does not let it know whether the previous save happened.

Testing the decisions as well as the answer

The baseline tests are deterministic: they replace the model with known responses and check what the harness allows to execute. This prevents a change in model wording from concealing a program error.

  • Before execution: invalid arguments produce a correctable error and the handler is not called.
  • When returning the result: a successful query remains successful even if the retrieved text mentions an error.
  • In each adapter: all three providers receive the expected format and the result is associated with the correct call.
  • During a save: a local exception neither confirms success nor triggers a second automatic save.
  • When keeping results: saving and retrieving a result preserves its success or error status without deriving it again from the text.

If a verification query is implemented, it needs its own tests: a meal actually saved, proven absence and an ambiguous result. Two similar meals must also be included to check that they are not confused.

A real-model test adds another check: whether the agent uses the result to continue the conversation. In Gymnasia, a missing-record query followed by another read was observed, as was a stop after a simulated save failure. The app was then reloaded to check the data instead of relying only on the visible answer.

This does not prove that an error flag always improves the model. In a comparison, the model also understood the absence from the old text. The demonstrated value of the contract is that the program recognizes the status and applies an explicit rule without depending on that interpretation.

Test scope and evidence

QA was performed on Gymnasia’s web app with OpenAI and fictitious data. The read request explicitly specified the alternative date. The save failure was injected into a weight record, not a meal: there was no further model request or automatic retry, and the record was absent after reloading. Once the fault was removed, a later save persisted correctly. Fake-provider tests cover all three formats; this real session is neither a native-device test nor an evaluation of every model. The verification report preserves the details and reproduction steps.

Building an agent, step by step

This article belongs to a series on building an agent and its harness using a gym app as an example. After declaring and validating tools, you need to decide how to continue when something fails: deliver a result that explains what happened, permit only the appropriate recovery and confirm effects before repeating them.

The pattern also applies to recording an order or creating a reservation. The domain changes, but the program still knows the execution result and controls what can happen next.

Explore the series index and the Gymnasia implementation used as the case study.

Continue reading

Last posts -->

Have you seen these projects?

Gymnasia

Gymnasia Gymnasia
Expo
React Native
TypeScript
OpenAI
Anthropic

Fitness app with two agents that run entirely on the device, with no backend, so the user's data never leaves the phone. A BYOK conversational coach with local tools and a remote system prompt with offline fallback, plus a food estimator that pulls macronutrients out of a photo of the plate, with barcode scanning against OpenFoodFacts.

LangGraph Deep Researcher

LangGraph Deep Researcher LangGraph Deep Researcher
Python
LangGraph
FastAPI
React
TypeScript
Docker

Multi-agent research system built with LangGraph. A supervisor breaks your question down into topics and launches search sub-agents in parallel; each one compresses its findings before handing them to a writer agent that produces the final sourced markdown report. Live streaming over WebSockets, a configurable model per role and bring-your-own API keys that are never persisted server-side.

Tau

Tau Tau
Python
LangChain

Multi-agent tutoring system for secondary school students, with one agent per subject and course material written and validated by a team of teachers. It was used with real students at a private school in Spain and at a secondary school in Colombia.

View all projects -->
>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Do you want to watch any talk?

Last talks -->

Do you want to improve with these tips?

Last tips -->

Use this locally

Hugging Face spaces allow us to run models with very simple demos, but what if the demo breaks? Or if the user deletes it? That's why I've created docker containers with some interesting spaces, to be able to use them locally, whatever happens. In fact, if you click on any project view button, it may take you to a space that doesn't work.

Flow edit

Flow edit Flow edit

FLUX.1-RealismLora

FLUX.1-RealismLora FLUX.1-RealismLora
View all containers -->
>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Do you want to train your model with these datasets?

short-jokes-dataset

HuggingFace

Dataset with jokes in English

Use: Fine-tuning text generation models for humor

231K rows 2 columns 45 MB
View on HuggingFace →

opus100

HuggingFace

Dataset with translations from English to Spanish

Use: Training English-Spanish translation models

1M rows 2 columns 210 MB
View on HuggingFace →

netflix_titles

HuggingFace

Dataset with Netflix movies and series

Use: Netflix catalog analysis and recommendation systems

8.8K rows 12 columns 3.5 MB
View on HuggingFace →
View more datasets -->