Stop 12 · When a tool fails
A call can fail in three ways, and in two of them you do not know whether the work happened — which is when a retry can do it twice.
Every model behaviour on this stop is a written illustration, not a live API call. The numbers are computed from the assumptions stated below.
exit 1, and stderr says connection refused.
You do. The connection was never made and the request never accepted; nothing happened on the far side.
Retry it, with the backoff from the next section. This is the easiest of the three to handle, and it is the one that looks the most alarming.
Three failures, not one
One of the runs at stop 2 was a tool failing, and that stop asked what to put in the . This one asks the other half: should your code try again? The answer depends on how it failed, and there are three ways, not one.
Open each of the three above and read the middle column — do you know whether the work happened? With a refusal you do: the request never got in, nothing ran, and retrying is safe. That holds only for errors that mean the request was not accepted — a server error such as a 500 can arrive after the work was done, and belongs with the other two. With the other two you do not. A timeout says you stopped hearing, not that nothing happened. The request may have arrived and completed with only the reply lost. And the one where your own side died is worse still: you may not even be sure it left.
All three look alike in a log — this call did not succeed — so the common treatment is to retry all of them once. For the first that is right. For the other two it can mean the work was done twice.
Beginner question: Does a longer timeout make this go away?
No, it moves it further away. Any finite wait has a moment where you give up just as the far side finishes, and the longer you wait the less patience the user and everything upstream has. You have moved the uncertainty from thirty seconds to five minutes. The fix is not on the waiting side but on the action side: make a repeated call unable to have a second effect.
| Attempt | Wait | total wait |
|---|---|---|
| 1 | 500ms | 500ms |
| 2 | 1.0s | 1.5s |
| 3 | 2.0s | 3.5s |
| 4 | 4.0s | 7.5s |
Retries need a limit and a gap
Two rules about retries barely need arguing. First, cap the number of attempts. Uncapped retries are the same object as the loop that would not stop at stop 2, except that this one does not even spend model tokens — it just flattens whatever it is calling. Second, wait between them, and wait longer each time (exponential ). The table above is 500 ms doubling.
The reason for backing off is worth stating: failures are often caused by the far side being overloaded, and an immediate retry adds to the overload. With a hundred clients all retrying immediately, what you have built is not a recovery but a second wave. In practice the wait also gets a little random jitter, so that everyone does not come back in the same millisecond.
| Tool | Called twice |
|---|---|
| search_files(query)search the project for a string✓ safe to repeat | Running it twice gives what running it once gave. Retrying needs no further thought. |
| read_file(path)read a file✓ safe to repeat | The same. Reads repeat safely, which is another reason stop 11 treats them differently from writes. |
| charge_card(amount)charge a card✕ not safe | Twice means charged twice. Retry blindly after a timeout and the customer is charged twice while your log shows one success.The fix: Give it an idempotency key: charge_card(amount, key: "order-8241"). The far side remembers it handled that key, so a second call carrying the same one returns the first result and charges nothing. The key is generated by the caller, tied to the piece of work, and reused unchanged on every retry. |
| send_email(to, body)send an email✕ not safe | The recipient gets two. Lighter than a double charge, and just as impossible to take back.The fix: The same key, or split sending into two steps: write to an outbox under a fixed message id, so that writing it twice still leaves one entry, and let a separate process that runs once do the sending. |
The real fix: make the second call do nothing
The second table above puts four tools side by side. The first two repeat harmlessly: searching twice gives what searching once gave, and so does reading. Tools like that are called — doing it many times has the same effect as doing it once. For them, retrying needs no design at all.
The last two are not. Charging twice charges twice; sending twice sends twice. On those, a blind retry after a timeout is a real loss, and it happens while your log shows a single success, which is the hardest kind to find.
The fix is to give the action an idempotency key: a string generated by the caller and tied to this piece of work, as in charge_card(amount, key: "order-8241"). The far side remembers it has handled that key, so a second call carrying the same one returns the first result and charges nothing. What matters is that a retry reuses the same key — generating a fresh one per attempt is the same as having none.
Put that back into the agent setting and it becomes a rule about tool design: the tools you give an agent are either safe to call twice or they carry a key. The reason is the one from stop 4: you cannot guarantee that the same agent run twice does the same thing, and you cannot guarantee it will not repeat a step after a timeout. Rather than telling it in the prompt not to double-charge — stop 6 covered how weak that is — make double-charging impossible.
Beginner question: Can the model decide whether to retry?
It can take part, and it cannot be the only thing deciding. The model sees only what you wrote into the tool_result (stop 2), so whether the work happened has to be put there by your code before it can judge correctly. And even when it judges correctly it is only likely to — stop 4 still applies. The workable split is that safety comes from the structure of the tool, and the model chooses among options that are already safe.
Either give an agent tools that are safe to call twice, or give them a key that makes the second call do nothing. Retries and backoff are the easy part; the hard part is knowing which actions can be repeated without consequence.
A look back: when it goes wrong
Three questions. Every idea in this group is one that looks solved when it is not.
Nothing is scored here. Every question can be answered again, and a wrong answer names the idea it came from.