Idempotent tools: a retry must not repeat what the agent already did

Idempotent tools: a retry must not repeat what the agent already did

One retry, two portions of rice

The user writes "Log 150 g of rice for today's lunch". Coach replies that it is done. The user opens their diary and the rice appears twice: 300 g and double the calories in the day's total.

Nobody asked for two portions and the model did not misread the sentence. The connection to the provider dropped halfway through the turn, the app retried the request and, in doing so, repeated a save that had already happened.

An operation is idempotent when running it once or several times leaves the same result. Reading the diary is idempotent by nature. Adding a food is not: each run adds another row. This article explains how to make a tool with effects behave as if it were idempotent, even when the whole turn is retried.

This is an illustrative example built on Gymnasia's real tool for adding foods. The automated test that protects it, described at the end, uses another tool with effects: the one that records a user suggestion.

Who asks, who executes and where it breaks

To see where the rice gets duplicated, follow the turn in order. The model executes nothing: it proposes a tool call, a request with the function name and its arguments. The harness, inside the app, validates it, runs it against the local data and sends the result to the model in a second request to the provider.

A turn with a tool that saves data
  1. The model asks to add the food.

    First provider response: a tool call with rice, 150 g and the meal.

  2. The harness saves the rice.

    Local action: the diary already has the row.

  3. The harness sends the result to the model.

    Second request to the provider. The network fails here.

  4. The app retries the whole turn.

    The model reads the user's message again and asks for the tool again.

Gymnasia retries up to three times. It cannot resume the conversation at step three, because the interrupted response is not kept, so it repeats the turn from the user's message. If nothing prevents it, step two runs again.

Retrying is right: networks fail and the user expects an answer. The defect is that the harness cannot tell "I am asked for a new action" apart from "I am asked again for the action I already did".

Why the provider's identifier is not enough

All three providers assign an identifier to each tool call. It lets the model match each result with its request: call_id in OpenAI, the id of the tool_use block in Anthropic and the call's id in Google.

The obvious idea is to store the identifiers already executed and skip the known ones. It does not work for this failure. On the retry, the model generates a new response and its tool call carries a new identifier. For the provider they are two different calls; for the user, the same action.

AttemptProvider identifierWhat it asks for
Firstcall_AAdd 150 g of rice to today's lunch
Retrycall_BAdd 150 g of rice to today's lunch

The conclusion is that the operation's identity cannot come from the provider. The harness has to build it from data that does not change between one attempt and the next.

An identity that survives the retry

These pieces stay the same between the first attempt and the retry:

  1. The user message that started the turn.

    Each message has its own identifier in the app, and the retry reuses the same one.

  2. The provider and the tool.

    Which API answered and which function it asks to run.

  3. The arguments, normalized.

    Keys are sorted before comparing, so {grams, food} and {food, grams} count as the same.

  4. The occurrence within the turn.

    How many times that same tool with those same arguments has already been requested in this turn.

The occurrence exists because repeating is sometimes legitimate. "Log one egg for breakfast and another for dinner" produces two calls with different arguments, because the meal changes. But if the model asks for exactly the same thing twice in one turn, the first is occurrence 0 and the second occurrence 1: two operations. On the retry, 0 and 1 come out again, and each one is recognized.

operation_id = hash(
  user_message_id,
  provider,
  tool_name,
  arguments_with_sorted_keys,
  occurrence_in_this_turn
)

Notice what is missing: the provider's tool call identifier. If the retry produces the same request, it produces the same operation_id, even if the provider renamed it from call_A to call_B.

Record before acting, confirm afterwards

With a stable identity, what remains is remembering what was done. That is the job of an operation ledger, kept in the app's persistent storage so it survives closing the app. Each operation with effects moves through states:

prepared

Recorded before running the tool: "I am about to do this".

committed

The effect was applied. The result the model received is stored too.

indeterminate

There is no proof of whether the effect happened. The next section explains what to do.

Order matters. If the operation were recorded after running it, the app closing between the two steps would leave an applied effect with no trace. Recording it first means you at least know it was in progress.

if ledger[operation_id] is committed:
    return ledger[operation_id].result   # replay, do not execute

mark operation_id as prepared
result = execute_tool()
mark operation_id as committed with result
return result

When the rice retry arrives, the identity matches and the ledger says "committed". The harness does not run the tool again: it sends the model the same result as the first time, the model writes its answer and the diary keeps a single row.

Returning the stored result matters as much as not executing. If the harness answered "it was already done", the model would receive something different from what it saw on the first attempt and could reply differently.

The ledger does not grow without limit. In Gymnasia each entry expires after seven days and it holds at most 256. When room is needed, only the oldest committed entries are removed, never one that is half done.

When you cannot tell whether the effect happened

The hard case is the gap between "prepared" and "committed". If the app closes right there, on return it finds a prepared operation and does not know whether the rice was saved. Two defenses complement each other.

  1. Let the data itself carry the identity.

    The food does not get a random identifier but one derived from operation_id. If the tool runs again, it finds a food with that identifier and adds nothing. It is a second safety net, in the data, in case the ledger's fails.

  2. Ask the data before repeating.

    Before running a pending operation, the harness checks whether its effect is already there. If it finds it, it marks the operation as committed without running it. On startup, the app does the same with every operation left half done.

That check has three outcomes:

It is there

The effect is found in the data. The operation becomes committed and is not repeated.

It is not there, provably

There was no effect. The operation may run.

It cannot be known

The operation stays indeterminate, is not repeated and the user is warned.

In the third case the harness does not guess. Gymnasia shows this message, in Spanish: "Gymnasia cannot confirm whether the action was completed. To avoid duplicating it, it has not repeated it. Check your data before requesting it again". Asking the user to check their diary is better than silently adding false data.

It also matters when the tool failed. If it fails before touching anything, for example because the arguments are invalid, the operation is removed from the ledger: there was no effect and retrying is safe. Only a failure that may have happened after the effect leaves the operation pending a check.

For a tool that writes outside the device, such as the one that records suggestions in an external service, the check means asking that service whether an operation with that identifier already exists. The pattern is the same: the harness needs a way to know whether the effect happened that does not depend on the response that was lost.

In the post on tool errors this verification appeared as a design option. Here it is part of the harness, and the program decides it, not the model.

Reads do not pay this cost

None of the above applies to tools that only read, such as "show me my routines" or "search for an exercise". Repeating a read changes nothing, so they are not recorded in the ledger and do not pay its extra write. Each tool declares whether it reads or has effects, and only the latter go through this mechanism.

How to test that nothing is duplicated

This failure depends on cutting the network at a precise instant, so testing it by hand does not work. You have to trigger it on purpose and check the final state, not just the visible answer.

  • With a simulated provider: the fake model asks for a tool with effects and the harness runs it. The second request to the provider returns an error and the app retries the whole turn. The fake model asks for the tool again, this time with another call identifier, as a real one would. The test checks that the effect exists only once, that the model received exactly the same result in both attempts and that the call identifiers were different. That last point proves the protection does not depend on the provider.
  • On the identity: the same arguments with keys in any order produce the same identity, and a second occurrence produces a different one. It is checked with many generated permutations, not a single example.
  • On the ledger: the operation is recorded as prepared before it runs, and a restart after the effect does not duplicate it. A prepared operation that cannot be checked never runs again, and one with proven absence does. A validation failure is not stored as a result.
  • On the limits: committed entries expire, there is a maximum number of entries and pending ones are never evicted. On collisions or an unreadable ledger, the harness stops instead of running blindly.

These tests prove the harness's local behavior. They do not reproduce a phone running out of battery in the exact millisecond between saving and recording. That case is what the indeterminate state is for: it does not promise to guess, only not to lie.

Building an agent, step by step

This article belongs to a series on how to build an agent and its harness using a gym app as the example. After running tools, returning their results and recovering from errors, one more guarantee remains: retrying must not multiply the effects.

The pattern works the same for creating an order or sending an email. Build the identity from what does not change between attempts, record the operation before running it and confirm it afterwards with its result, and decide in advance what to do when you do not know whether something happened.

See the series index and the Gymnasia implementation used as the case study.

Continue reading

Last posts -->

Have you seen these projects?

Gymnasia

Gymnasia Gymnasia
Expo
React Native
TypeScript
OpenAI
Anthropic

Fitness app with two agents that run entirely on the device, with no backend, so the user's data never leaves the phone. A BYOK conversational coach with local tools and a remote system prompt with offline fallback, plus a food estimator that pulls macronutrients out of a photo of the plate, with barcode scanning against OpenFoodFacts.

LangGraph Deep Researcher

LangGraph Deep Researcher LangGraph Deep Researcher
Python
LangGraph
FastAPI
React
TypeScript
Docker

Multi-agent research system built with LangGraph. A supervisor breaks your question down into topics and launches search sub-agents in parallel; each one compresses its findings before handing them to a writer agent that produces the final sourced markdown report. Live streaming over WebSockets, a configurable model per role and bring-your-own API keys that are never persisted server-side.

Tau

Tau Tau
Python
LangChain

Multi-agent tutoring system for secondary school students, with one agent per subject and course material written and validated by a team of teachers. It was used with real students at a private school in Spain and at a secondary school in Colombia.

View all projects -->
>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Do you want to watch any talk?

Last talks -->

Do you want to improve with these tips?

Last tips -->

Use this locally

Hugging Face spaces allow us to run models with very simple demos, but what if the demo breaks? Or if the user deletes it? That's why I've created docker containers with some interesting spaces, to be able to use them locally, whatever happens. In fact, if you click on any project view button, it may take you to a space that doesn't work.

Flow edit

Flow edit Flow edit

FLUX.1-RealismLora

FLUX.1-RealismLora FLUX.1-RealismLora
View all containers -->
>_ Available for projects

Do you have an AI project?

Let's talk.

maximofn@gmail.com

Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.

Do you want to train your model with these datasets?

short-jokes-dataset

HuggingFace

Dataset with jokes in English

Use: Fine-tuning text generation models for humor

231K rows 2 columns 45 MB
View on HuggingFace →

opus100

HuggingFace

Dataset with translations from English to Spanish

Use: Training English-Spanish translation models

1M rows 2 columns 210 MB
View on HuggingFace →

netflix_titles

HuggingFace

Dataset with Netflix movies and series

Use: Netflix catalog analysis and recommendation systems

8.8K rows 12 columns 3.5 MB
View on HuggingFace →
View more datasets -->