The prose of all fourteen stops, in reading order, on one page. Every heading is anchored, so a link can point at a paragraph.
This page is for coming back. Having read the course once, you usually want one paragraph again rather than fourteen stops. The parts of each stop you operate are not on this page; the other four runs in Watch it run and the check at the end of each group appear as text only, and are marked.
If you have written print("hello world") once, you already know what a program is: a list of instructions for the computer, followed one line at a time. That is all. Every new idea used here — , , — is explained at the moment you need it, and any word with a dotted underline can be clicked for a plain explanation. Beyond that one line of print, you do not need to be able to write code — though a dozen or so ordinary programming words are used without a definition, and the README on the course's GitHub page lists them.
The technology behind ChatGPT and Claude is called a large language model. Its entire ability fits in one sentence: you give it text, and it produces more text. There is no magic and no consciousness behind it. Watch the animation below: a question goes in, an answer comes out. That is the only thing it does.
The moment you ask it to do something real — read a file on your computer, fetch a web page, run a command — the limit shows. The model runs on someone else's server and cannot touch your computer at all: no files, no network, no keyboard. A model that only produces text cannot do the work for you. This is the line between a chatbot and an agent. (The chat products you have seen search the web or read files are doing exactly what the next scene describes.)
The fix is simple. First you tell the model which tools exist: read_file, run_command, and so on. When it needs one, it returns a structured request naming the tool and its arguments — this is called a tool_use block. The code on your own computer runs the tool. Then you send the result back as a new message for the model to read. Remember the split: the model asks, your code acts.
Wrap that exchange in a loop: send the array to the model, append its reply to the array, run any tool the reply asks for and append the result too, then send the array again. The loop repeats until a reply comes back with no tool request in it. You never set the number of rounds in advance — the loop keeps going as long as the replies keep asking for tools. A program built this way is what people call an agent. Watch two rounds go by below; the rhythm is these four steps.
The whole conversation history — your words, the model's replies, the tool results — lives in one . One loop keeps repeating the same four moves: send the array, append the reply, run any requested tool, append the result. Claude Code and Cursor look complicated, but at the core they are larger versions of exactly this. The next two stops: first watch it run in slow motion, then write it yourself.
The panel on the right is an X-ray: it shows the real data that will be sent to the , not an interface built for people. Right now the is empty — nothing has happened yet. The at the top is the job description for the model (who it is, which tools it may use, which rules to follow). In code it is a separate parameter, sent along with every request. Remember this picture: everything that follows is just appending elements to this array.
Why does the array matter so much?
The API keeps no state between calls: every request is handled as if it were the first. What people call the conversation, the context, or the memory is literally this array. You keep the full history yourself and hand the whole thing over every time. Once you understand this, an agent is no longer mysterious.
What you type in the chat box becomes one in code: the { role: "user", content: "…" } is appended to the array. Note the key detail: nothing has been sent to the model yet. Putting a message into the array and sending the array to the model are two separate actions, and only the first one has happened.
What is role? How many are there?
Every element has two fields: role (who it came from) and content (what it says). Only two roles appear in this array: assistant, for what the model itself produced, and user, for everything else — your words and the tool results your code produces. There is no tool role — in a moment you will see that tool results also travel inside a user message.
Now the whole array, together with the system and tools parameters, is sent to the . This time the reply is not a final answer: it is a tool_use block, which means "please run ls for me". The key point is that the model only returns a request. It has no hands and can execute nothing. : tool_use means it is your code's turn to act. This reply is pushed into the array as-is, as an assistant message.
How does the model know which tools exist?
From the tools parameter in the create call: you send a list giving each tool a name, a purpose, and a parameter format. The model can only ask for tools on that list, and your code is always the one that runs them. Send no list, and it has no tools.
Your code checks stop_reason first. It is tool_use, so instead of leaving the loop the code actually runs ls. This step happens entirely on your machine: the model cannot touch your file system, and it cannot see whether the command ran — it only sees the text you send back. When the command finishes, the output is wrapped as a and pushed into the array as a user message. To the model, a tool result is no different from anything else it is shown: it is just text.
Why is the tool_result role user, not tool?
The Claude API has no tool role. The rule is simple: only what the model itself produced is assistant, and everything else is user — the words you type, and the tool results your code produces.
The loop returns to the top and runs the same create line again, resending the whole array — now longer — from the very beginning. The API is : it keeps nothing from the previous round, so "memory" just means you carry the full history every time. This round the input contains the ls output, and the reply asks for read_file on package.json: another tool_use. Watch the running count in the corner: the longer the array, the more each round sends, and the faster the total climbs.
Is resending everything wasteful?
Yes, and this is exactly why an agent gets more expensive the longer it runs. Real systems reduce the cost with prompt caching, which discounts a repeated prefix, but the basic model does not change: the full array goes out every round. Understand this and you understand why long conversations become slow and expensive.
The same routine again: run it locally, wrap the output as a tool_result, push it into the array. The pattern should be visible by now. An agent has no magic step; it mechanically repeats one rhythm: send the array, check whether a tool is requested, run it, append the result. In code, that rhythm is the while (true) loop — the lines highlighted below are the same ones as before.
What if a tool fails?
Put the error message into the tool_result exactly as it is and send it back (the Claude API also lets you mark that tool_result with is_error: true, saying plainly that it failed). On the next round the model sees that error, and its reply can retry with different arguments or take another route. That is the whole mechanism behind agents that appear to correct themselves: every round, the true result of the previous step is in the array, including failures.
The whole array goes out a third time. The array now holds enough information, so the reply is plain text with no tool_use block, and stop_reason becomes end_turn. The if condition in the code finally holds, break leaves the loop, and the run is over. Look back at the whole thing: one array grew from empty to 6 elements, one loop ran 3 rounds, and you ran 2 tools on the model's behalf. That is all an agent is: one array and one loop.
So how much code do I actually have to write?
The 32 lines in the panel below are the complete skeleton of the loop, and nothing in the loop itself is left out. Only tools and runTools are written elsewhere: how to write tools is the subject of stop 7, and runTools just hands each request to the code that handles it. At the next stop you fill it in one blank at a time, and you will find that you have already pressed a button for every one of these lines on this page.
The skeleton is ready. You fill in the 8 key blanks. Each blank teaches the concept it needs before it asks, and a wrong answer gets a specific explanation.
An array = a row of numbered slots that you can keep appending to. It is written with square brackets: ["apple", "banana"] has two slots, and [] is empty — no slots yet. Our agent uses one array as its memory.
Correct. This array is the entire memory of the agent, and everything that follows just appends to it.
An object = a labeled box, written { label: value }. One message is one object: the role label records where it came from, and the content label records what it says. Text (a string) needs quotes, like "hello".
Correct. Everything you write has the role "user". Remember this word — the last blank asks for it again.
A loop = making the computer repeat something. while (condition) { … } means: as long as the condition holds, run the braces again from the top. The condition is a boolean, which has only two values: true and false.
Correct. while (true) never stops on its own. How many rounds it runs depends on the replies, and break is what gets you out.
A variable = a name you give something. On line 5 the array was named messages, so writing messages anywhere after that refers to the same array. The name alone is enough here: when a parameter and a variable share a name, messages: messages can be shortened to messages, which is also how the tools line above is written.
Correct. Every round resends the whole messages array. The API keeps no state between calls, so this array is the memory.
The dot (.) = reaching into a box. The parcel that comes back is called res, and it has several compartments inside. res.content means the content compartment of res.
Correct. res also carries metadata such as stop_reason and token usage. Only the content itself goes into the array.
break = leave the loop immediately. No more rounds run, and the program continues after the loop. It is the only normal way out of a while (true) loop.
Correct. break is the only normal way out of this infinite loop.
No new concept here. This blank tests the point people most often get wrong at stop 2. Think of the single rule.
Exactly right. Only what the model itself produced is assistant; everything else is user: your own words, and the tool results your code produced. You now know every key point.
Above is one task: count the TODO comments left in this project. The is identical, the system prompt is identical, the tools are identical. Press Run it again a few times and watch what it decides to do.
Most of the time it searches straight away, sometimes it lists the directory first, sometimes it guesses a path that does not exist, and occasionally it just states a number without looking. Two of those four reach the right answer, one costs an extra round, and one produces a figure with nothing behind it — and you changed nothing.
A model does not think of the next word. What it computes is a table: for every possible that could come next, how likely it is. Something then has to pick one out of that table. That step is called . It is not the network itself but a step the inference service runs after it; through an API it happens on the server, and your code can only influence it through parameters.
is the knob on that step. It does not change the table the model produced; it changes how the pick is made. Low temperature and the most likely token is chosen nearly every time; high temperature and the ones further down get a turn. Drag the temperature to 0 above and press again a few times: the same path every time. Drag it to 1 and the path that answers without looking starts to appear. (The slider shows the principle. Some models' APIs no longer expose temperature at all, and then the variation can only be narrowed through the prompt and the tool results.)
The part that matters is that this choice happens once per token generated. A reply is hundreds of tokens long, and one different pick early on carries everything after it somewhere else: choose "src" rather than "." as the third token and the whole run becomes a different run.
Does temperature zero make it fully deterministic?
Closer than you would think, and not quite. First, even with deterministic sampling the arithmetic on the server is not guaranteed to be bit-identical across batches and hardware, and two probabilities that sit very close together can swap. Second, and more importantly: an agent's input is not only your prompt, it is also whatever the tools returned. A file changed, a command was slower, a page was edited — the array is different, so the rest is different. Temperature zero removes the randomness in one place; it does not remove the fact that the agent runs in a world that moves.
This is the thing to carry away. A function in the mathematical sense — a pure function, in programming — has the property that the same input gives the same output: test it once and you know how it behaves. An agent does not have that property. The same input can take a different path, which makes "it worked when I tried it" not evidence but one sample.
That sentence decides how the later stops work. Why a prompt change is checked against a whole set of tasks rather than the one that annoyed you (stop 13): because one pass says very little. Why the same task is run three times and recorded as how many of three passed: because a pass rate is a comparable quantity and a single result is not. Why production shows behaviour you never saw: because what you saw were the likeliest paths, and your users are sampling the rest on your behalf.
How do you debug something that is not deterministic?
Move your attention from why this run was wrong to how often this kind of run is wrong. In practice: run it several times and record the distribution rather than trying to reproduce one case; save the full array from each round, so that at least that path can be walked again afterwards; and keep the uncertainty away from the edges — approval for anything with a side effect (stop 11), and tools that are safe to call twice. You cannot remove the randomness. You can make being wrong cheap.
An agent is not a function. The same input can take a different path, so "it worked when I tried it" is not evidence — it is one sample. Turning the temperature to zero reduces the variation without removing it, because the environment differs between runs as well.
What the three questions above have in common is that none of their answers could be inside the model. The first changes by the minute, the second lives in your repository, and the third does not exist. Switch to without tools and all three replies are fluent, specific and confident, and none of them is reliable.
Look at the third one in particular. The question presupposes that formatMoney exists, and the model continues from that presupposition into a perfectly plausible description. It did not hedge, and it was not lying. It was doing the thing it always does: continuing a passage of text.
Back to the sentence from stop 1: the model's entire job is to take text and produce more text. What decides the next is how well it fits after this context, not whether the resulting sentence is true. Fitting well and being true are strongly correlated in training and are not the same thing — and on anything it has not seen, they come apart.
So the invented sentence was produced by the same process as a true one. There are signals inside a model that track whether it knows something, but they are unreliable, and nothing makes them stop an invented sentence on its way out. Which explains a common disappointment: asking are you sure usually achieves little. What comes back is mostly a well-fitting passage about certainty, not a second look — it has nothing to look at except what it just wrote.
Can you not just tell it to say when it does not know?
It helps, within limits, and the reason is worth understanding: the instruction raises how willing it is to say I do not know, but it cannot make the model reliably tell what it knows from what it does not. So it reduces invention without removing it; sometimes it also makes the model decline questions it would have answered correctly, while the wrong answers that felt familiar still come through. Pair the sentence with a place to look and it works much better.
stop 1 said tools exist so the model has hands: to read a file, to run a command. Here is the other half, and the more fundamental one: tools exist so it has somewhere to look. A model asked what time it is has to guess. A model with a clock does not. The difference is not that it became smarter; it is that the answer moved from an impression about text to a tool_result in the .
The third question makes this clearest. The most underrated thing a tool does is not to supply a fact but to supply the absence of one: search_files("formatMoney") returning nothing is itself evidence that can be written into the array, and that is what gives the model grounds to say the function does not exist. Without a tool, "I do not know" has no shape, because nothing is pushing it in that direction.
The usual next move is to demand sources: retrieve the material into the array and have the answer cite it. That works — it is usually called retrieval-augmented generation, or RAG — and it is the most practical technique available. It has one boundary worth being clear about: a citation is also generated text. A filename it never read, or a line number assembled to look right, is indistinguishable from a real source.
So the technique that pays is not requiring a citation but requiring a citation you can check, and then checking it. A reference to lib/client.ts line 4 is worth something because opening the file falsifies it in seconds. A reference to the project documentation is worth nothing, because it points at nothing you can open. A tool that returns a filename and a line number makes verification a few seconds' work — how to say in a tool's description what it returns is covered at stop 7.
Does retrieving the material stop it inventing?
Much less, not none, and there are three gaps worth remembering. When retrieval finds nothing relevant the model still answers, unless your tool says plainly that nothing came back. What was retrieved can itself be wrong or out of date, and the model is in no position to doubt it. And retrieved material is outside content, which may be talking to the model — that is stop 10. Retrieval replaces inventing from impressions with depending on what you supplied, which makes the quality of what you supplied the new ceiling.
The model is continuing a passage, not consulting a record, so "I do not know" has no natural shape for it. A tool gives it that shape: I looked, and nothing came back, therefore there is none. Do not ask whether it knows. Give it somewhere to look.
The code panel at stop 2 has this line: system: "You are a local file assistant…". It is a parameter alongside messages, not an element of the . The distinction is deliberate: the array grows with the conversation, gets truncated, gets summarised (stop 9), while the system prompt is the passage you supply again, unchanged, on every round. It cannot be squeezed out by a long conversation.
Supplied again, unchanged, on every round has two consequences. The good one is stability: however long the conversation runs, that passage is still there and still first. The other one is the bill: it is charged again every round.
Apply the rule seen at stop 2 — everything is resent every round — to the system prompt: a 900- passage in a forty-round run goes out forty times. You wrote it once and bought it forty times — the figure above is computed with the same cost model stop 8 uses.
This is not an argument for short prompts. Nine hundred well-spent tokens usually save far more than they cost. What is worth remembering is the other thing: every sentence in a system prompt is billed per round, so a paragraph added just in case costs you not once but every time. And a related point: it already sits ahead of the conversation, so it earns the cache discount from stop 8 as long as it stays byte-identical — drop a current timestamp into it and the whole prefix stops matching.
Do the long you must follow these rules lists work?
They work, with returns that fall off quickly and failures that are hard to see. For a rule to take effect it has to win, on every round, against every opposing pressure in the context at that moment — the wording of the task, the contents of a tool result, and the sampling from stop 4. Ten rules that matter usually outperform forty. And the one or two that matter most are better not implemented as rules at all, which is the next section.
The three conditions above run the same task. In the first, nothing was said and the mail went out — and no step was taken wrongly: the task says send it to me and the tool list contains send_email. In the second, a sentence was added to the system prompt and nothing went out this time. In the third, every tool that could send anything — send_email, and run_command, which runs anything — was removed from the list and nothing could go out.
The second and third produce the same outcome and are not the same kind of thing. The second is a passage of text sitting alongside other passages of text: it has to win again on every round against the task's wording, the tool results, and the sampling from stop 4. It wins nearly always, and nearly always is not always. The third enters no contest at all — a tool that is not in the list cannot be requested, regardless of what the model was thinking or what it had just read.
stop 10 takes this to its limit: if a piece of outside content can address the model, then I told it in the system prompt becomes a contest with an attacker over whose text is more persuasive. You will not necessarily lose that contest, and you will not necessarily win it, and you do not know in advance. So the order of the questions should be: first ask whether the thing can simply be made impossible, in which case no instruction is needed; only when it has to remain possible do you fall back to moving a tendency with a prompt.
A system prompt moves a tendency; the tool list decides what is possible. "I told it to be careful" is an order of magnitude weaker than "I did not give it that tool" — the first has to hold again on every round and against every piece of outside content, while the second, once every route is closed, holds once and holds for good.
The tools parameter you send to the is a list, and each entry has three parts: a name (what the model writes when it asks for it), a description (a passage of ordinary language), and an input_schema (the shape of the arguments, as JSON Schema). The name and the schema have two readers: the model asks in that format, and the name is also one of its clues for choosing; your code dispatches and validates against them.
The description is different. It has no required format, takes part in no validation, and changes no line of your program. Its one reader is the model, and its only effect is to change what the model picks. In other words, it is a prompt. A sentence added here does the same kind of work as a sentence added to the system prompt — while a comment above the function body is never seen by the model.
How long should a description be?
It costs s like any other prompt, and it is resent on every round (stop 8), so longer is not better. A workable bar: write until someone who has never seen the project could decide correctly, from that passage alone, whether to use it. Two or three sentences usually do it — one for when to use it, one for the boundary, one for what comes back and how much.
The three tasks above correspond to three omissions. What they have in common: every mistake in the left-hand column traces back to the description. The model reasons from an incomplete description, and the reasoning is not far-fetched. (In the second task its claim to have made the change is invented — but nothing in the description warned it that the toolset only reads.) That is also why a stronger model usually does not fix this class of problem: what is missing is not reasoning, it is information.
There is a practical test that requires running nothing: copy the descriptions out of your tool list, on their own, with nothing else, and ask yourself whether you could answer, for each tool, when to use it, what it does not do, what comes back, and how much. Wherever you cannot, the model cannot either — and it will not tell you that it is guessing.
What if two tools overlap?
First ask whether they should be one tool. If two are genuinely needed, write the boundary into both descriptions as a rule that can be followed, and have them name each other: "for questions about this project use search_files, not search_web", and "search_web is only for information that could not be in this project". Describe each in isolation and the model re-guesses the boundary every round — and it will not necessarily guess the same way twice.
Describing a tool means writing down four things: when to use it, what it does not do, what comes back, and how much. Leave one out and the model will assume on your behalf — and its assumption is always that a limit you did not write down does not exist.
stop 2 showed this as a fact about how the API works: it is , so every round resends the whole from the beginning. Now treat it as arithmetic. Round 1 sends the system prompt, the tool list and your task. Round 2 sends all of that plus the model's reply plus the tool result. Round 3 sends all of that plus another round's worth.
The second step is the one that matters: you do not pay for the last round, you pay for the sum of all of them. In a run that adds six hundred s per round, round 40 sends about twenty-five thousand tokens — but rounds 1 through 40 together send over half a million. What grows linearly is the size of one round. What grows quadratically is the total. The bill is the total.
Why the square?
Line up the size of each round: a, a+g, a+2g, …, a+(n−1)g. That is an arithmetic series, and it sums to n·a + g·n(n−1)/2. The second term contains n², which dominates once n is large. In plain words: by round 10 you have sent the first message ten times, and by round 40 you have sent it forty times. The number of rounds is growing and so is the number of times the early content is re-bought. Two things growing together multiply.
Almost everything sent on a round is identical to the previous round; only a short piece at the end is new. Prompt caching exists for exactly this: if a round begins with the same content as the last one, that prefix can be re-read at a fraction of the normal price, and only the new part is paid for in full, plus a little more to write it into the cache. Turn the switch on above and watch the second curve.
Look carefully at the shape: the cached line still bends. The re-read prefix is billed at a tenth, but it is still billed once per round, so the curve is still quadratic with a much smaller coefficient. Under the assumptions on this page, forty rounds fall from $1.68 to $0.36 — about 4.6 times cheaper, not ten times, because output tokens are still charged in full and each round's new slice is charged at the cache-write rate, a quarter above it. Caching is a discount, not an exemption. An agent that runs five hundred rounds is expensive either way; with caching it is less expensive.
Is caching always worth switching on?
Almost always, with two conditions. First, what is cached has to be a prefix: insert one sentence at the start of the array and everything after it shifts, invalidating the whole cache. So a random greeting or the current timestamp does not belong in the system prompt. Second, writing to the cache costs a little extra, so a conversation that ends after a single round comes out about 14% more expensive with caching on. Drag the round count to 1 and the cached line sits above the other one; drag it to 2 and it is already below.
Turn on the second switch above: paste a twenty-thousand-token document into the first message — a specification, a log, a large JSON blob. It is pasted once, but it sits at the front of the array, so it appears in every request from then on. Over forty rounds you pay for that document forty times: six cents of tokens becomes $2.40, and the run goes from $1.68 to $4.08. In thirty-nine of those forty it may not have been needed at all.
The rule this gives you is concrete: keep bulk behind a tool and fetch it when the model actually asks, rather than loading it up front. The instinct to put everything that might be relevant into the context and let the model pick is, in a metered loop, the most expensive thing you can do.
Whatever you put into the on round 1, you buy again on every round after it. The thing to control is not how many rounds a run takes, but how much each round has to carry.
The context window is a hard ceiling. There are four ways to deal with it — three once it is full, and one that keeps material out in the first place — and all four lose something.
A model does not measure text in characters or in words but in s. Split a sentence the way a model actually sees it and you get something like: "Find / out / why / the / order / tot / als / are / wrong" — common words are one piece, unusual ones get broken up, and punctuation and spaces take up room too. In English a token is roughly four characters; in Chinese the ratio depends on the tokenizer — a common word of one or two characters may be a single token, while a rare character can take several.
Why it matters: there is a ceiling on how many tokens a model can read in one request, and that ceiling is the . It is not a soft limit that makes things slow. It is a wall — one token over and the request is not accepted at all. The 12 MB log that failed at stop 2 hit this wall.
The windows are large. Does anyone actually run out?
In ordinary chat, almost never. In a tool-using agent, easily: a directory listing, a few files, a test run, a stack trace, and a dozen rounds puts you in the tens of thousands. Add someone pasting a PDF or a whole schema into the first message and it goes fast. And remember stop 8: staying under the ceiling is not the same as being cheap. Everything you keep in the window is paid for again every round.
The conversation above has run three rounds and comes to 1,150 tokens, against a window of 1,000 in this example. The next round cannot be sent. There are exactly three things to do about it; try each with the buttons above:
Refusing is the most honest and the least useful: tell the user it does not fit and stop. In many products that is the right call — better than an answer built on a mutilated history. Truncating is the most common and the easiest way to fail quietly: drop from the oldest end. Try it and watch what goes first — the task itself, after which the model is answering a question it can no longer read. What is left also starts with an assistant message, which many APIs refuse; real truncation has to keep a user message first. Summarising costs the most and helps the most: spend an extra model call folding the early stretch into a short passage and put that back. It keeps what the job was, at the price of detail that is gone for good — the summary no longer says that config.json sets two decimal places, or that total used to be DECIMAL(10,2), and nothing can recover either.
Why not just drop the big ones in the middle?
You can, and in practice people do, but be careful: messages in the array are not independent. An assistant tool_use and the tool_result after it are a pair. Drop the result and keep the request, and the model sees itself asking for something that never came back. Keep the result and drop the request, and an answer appears that nobody asked for. Drop them in pairs, and make sure what is left still reads as a coherent history.
The three ways above all trade inside one conversation: what stays and what goes. There is a fourth, which is that some material should never enter this at all. "Read the 30 files under lib/ and find the ones that touch authentication" is the shape: thirty files come in and are resent every round afterwards (stop 8), and what you wanted was a list of four or five names.
The move is to start a second agent with its own array, its own loop and its own budget. It reads the thirty files across thirty-one rounds and hands back only the conclusion. On the caller's side the whole episode is one tool call: one message out, one message back. The two columns above are two ledgers for the same event — the caller's array grew by about 400 tokens while the episode cost a hundred and twenty-eight thousand. What was saved is not those hundred and twenty-eight thousand but the fact that they are not resent on every later round.
The price is in the last two rows, and it is a different kind of loss from the first three. Truncation loses content, and you know what you lost. Handing work off loses the ability to check: the caller holds a 300-token passage and not the thirty files, so it has no way to verify that passage. This summary did miss a file — the subagent read lib/cache.ts truncated and never mentioned it — and what reached the caller was a list that reads as complete.
This is the shape from stop 2 again, one level up: there a tool failed to report a failure, and here an entire agent failed to report a truncation. Every boundary you add is one more place information can quietly disappear, and the outer layer never sees inside the inner one. The workable middle is to have the subagent return a few lines of metadata alongside its conclusion: how many files it read, whether anything was truncated, whether a tool failed, and where it was unsure. Those lines cost almost nothing and turn a silent omission into a visible doubt, which is something you can act on.
When is handing work off worth it?
Three questions: does it read a lot, is the conclusion short, and would a mistake be visible? Reading thirty files and returning a four-line list answers yes to all three, and is worth it. Editing three files fails the third — a bad edit announces nothing, and the caller holds no original to compare against. Put the other way round: when a mistake would fail quietly, do not put a boundary in front of it.
A full context is not a problem intelligence can solve. It is a question your code has to answer: what gets thrown away. Refuse, truncate, summarise, or hand the work to another agent — each loses something different, and where the first three lose content the fourth loses the ability to check. Declining to choose is choosing the worst one, where the request is simply refused.
The task is ordinary: summarise this vendor's pricing page. The agent calls fetch_url, appends the page as a tool_result to the , and summarises. So far this is no different from reading package.json at stop 2.
The switch above shows the same HTML two ways: as a person sees it in a browser, and as it enters the array. A browser hides the div that has been moved off-screen; a fetching tool does not — it takes all the text in the HTML and hands it to the model unchanged. Switch over, look at it, then read what the model produced next.
This is called . What makes it hard is not that the model is not clever enough; it is that the only thing separating "this is my instruction" from "this is data from outside" is a label. Recall the rule from stop 2: only user and assistant appear in the array, and tool results travel as user — the same role as the words you type yourself, only wrapped in a tool_result block. Models are trained to be wary of instructions inside such blocks, but that is a tendency: by the time they reach the model, your task and that hidden passage are both text, and a passage written convincingly enough as an instruction can still be taken as one.
So this is not a problem that disappears with a stronger model; it is a property of the data structure. As long as an agent puts outside content into its context, it can read a passage that is trying to direct it — and outside content includes web pages, email, issues, pull request descriptions, logs, filenames, code comments written by someone else, and any string you do not control.
Would a separate tool role fix it?
No, though the instinct is a reasonable one. Some APIs do have a separate tool role, and the Claude API marks tool output with a tool_result block, but these are labels: what reaches the model is still one continuous sequence of text, and a label makes the model more careful without stopping a persuasive enough passage. The label makes "do not take orders from this part" easier to state, so it helps — but it helps at the level of prompting, not isolation. Isolation comes from what the agent is able to do.
Open each of them above. Note the split: the two marked "wording" both come down to having a model read a few more sentences and hoping it listens to you rather than to the attacker. They lower the odds, but the attacker is writing the same kind of thing, so they are never a guarantee. The three marked "structural" change something else: not who the model believes, but what it can reach and what it can accomplish even after being persuaded.
In practice you stack them, and you design as though the injection will succeed: ask what the worst outcome is if the model does exactly what that passage says, then remove that worst outcome structurally. That question is far more useful than asking how to keep the model from being fooled, because it does not depend on guessing how the attacker will write.
Treat anything from outside that enters the as capable of speaking to the model. Wording lowers the odds; what actually holds is structural — what data this agent can reach, and what actions it can take.
Look at the first beats of the run above: read_file simply ran, and write_file stopped. Where that line is drawn is worth stating plainly: it is drawn at whether there is a side effect, not at how dangerous something is.
The two get conflated, and they are not the same. Reading a file can be extremely dangerous: read .env and its contents enter the , where they are resent with every later round (stop 8). Writing a file can carry no risk at all: a line appended to a log in a temporary directory. But whether a tool has a side effect can be labelled when the tool is defined — read_file has none, write_file does — while whether a call is dangerous depends on its arguments and its moment and cannot be labelled in advance. Drawing the boundary at the question that can be answered beforehand is the version of this that can actually be built.
The more useful version of the question is reversibility: if this step were wrong, could I take it back? Reading changes nothing in the outside world, so in that sense it has no side effect; but what it read has already been sent to the model and cannot be taken back, which is why reads are governed by scope (see the question below). Writing a file is nearly reversible under version control. Sending an email is not. Moving money is not. Sort your tools by reversibility and the set that genuinely needs a human nod turns out to be much smaller than instinct suggests.
So reads need no control at all?
They need control, of a different kind. Side effects are governed by approval; reads are governed by scope. Deciding which directories this agent can see at all is worth far more than a dialog before every read, because the first is a rule you set once and the second is a button you will get tired of. They solve different problems, and using one for the other usually does neither well.
The three buttons above are, in most systems, the whole menu. They look like three levels of strictness. They are three different promises, and the middle one almost always promises more than people think.
Allow always usually binds to the tool, not to the arguments in front of you. You clicked it while looking at write_file("lib/cart.ts"), so what you approved was write_file — and the later write_file("package.json") no longer passes through you. This is not an oversight in the implementation; the word always simply has no scope of its own. The scope has to be supplied by whoever designs it: a directory, a shape of argument, a window of time.
People expect a refusal to make the agent crash or stall. It does not. In the array a refusal is an ordinary tool_result, structurally identical to a failed command or a missing file. The rule from stop 2 applies again: what the model can think of next depends on what that message says.
So a refusal deserves to be written well. Denied on its own tells the model only that this road is closed; it may try another tool, or give up with a vague account of why. Denied: the user did not approve this write, you may output the change as text instead gets you the diff — the task is finished, only delivered differently. A good refusal answers two things at once: no, and here is what you can do.
Directly: approval exists because the text asking for the action may not have come from you. The page at stop 10 can make an agent request an action, and the agent cannot reliably tell which sentence was yours. Approval is the remedy for exactly that confusion — it does not judge who asked, it only insists that something with a side effect gets a nod. Which is also the argument for keeping the scope of an always small: every inch you widen is an inch an injection can use.
An agent that asks about everything goes unused: after thirty confirmations a person writes a script to get around it. An agent that asks about nothing goes untrusted: the first time it deletes the wrong thing, it never touches production again. Every real system sits somewhere on that line, and where depends on the setting — running tests locally and sending mail to customers should not share a threshold.
A few things hold wherever you sit. Approve actions, not tool names. Show the actual arguments in the prompt, or the person is approving write_file rather than approving writing that file. Give always a scope and an expiry. And the one that matters most: if an action cannot be undone when it is wrong, it should not be guarded by a single click. Make it undoable first — stage it, produce a diff, put it in drafts.
An agent that asks about everything goes unused; an agent that asks about nothing goes untrusted. Every real system is a position on that line. The way to choose one is not to ask how risky a step is, but whether you could undo it if the step were wrong.
One of the runs at stop 2 was a tool failing, and that stop asked what to put in the . This one asks the other half: should your code try again? The answer depends on how it failed, and there are three ways, not one.
Open each of the three above and read the middle column — do you know whether the work happened? With a refusal you do: the request never got in, nothing ran, and retrying is safe. That holds only for errors that mean the request was not accepted — a server error such as a 500 can arrive after the work was done, and belongs with the other two. With the other two you do not. A timeout says you stopped hearing, not that nothing happened. The request may have arrived and completed with only the reply lost. And the one where your own side died is worse still: you may not even be sure it left.
All three look alike in a log — this call did not succeed — so the common treatment is to retry all of them once. For the first that is right. For the other two it can mean the work was done twice.
Does a longer timeout make this go away?
No, it moves it further away. Any finite wait has a moment where you give up just as the far side finishes, and the longer you wait the less patience the user and everything upstream has. You have moved the uncertainty from thirty seconds to five minutes. The fix is not on the waiting side but on the action side: make a repeated call unable to have a second effect.
Two rules about retries barely need arguing. First, cap the number of attempts. Uncapped retries are the same object as the loop that would not stop at stop 2, except that this one does not even spend model tokens — it just flattens whatever it is calling. Second, wait between them, and wait longer each time (exponential ). The table above is 500 ms doubling.
The reason for backing off is worth stating: failures are often caused by the far side being overloaded, and an immediate retry adds to the overload. With a hundred clients all retrying immediately, what you have built is not a recovery but a second wave. In practice the wait also gets a little random jitter, so that everyone does not come back in the same millisecond.
The second table above puts four tools side by side. The first two repeat harmlessly: searching twice gives what searching once gave, and so does reading. Tools like that are called — doing it many times has the same effect as doing it once. For them, retrying needs no design at all.
The last two are not. Charging twice charges twice; sending twice sends twice. On those, a blind retry after a timeout is a real loss, and it happens while your log shows a single success, which is the hardest kind to find.
The fix is to give the action an idempotency key: a string generated by the caller and tied to this piece of work, as in charge_card(amount, key: "order-8241"). The far side remembers it has handled that key, so a second call carrying the same one returns the first result and charges nothing. What matters is that a retry reuses the same key — generating a fresh one per attempt is the same as having none.
Put that back into the agent setting and it becomes a rule about tool design: the tools you give an agent are either safe to call twice or they carry a key. The reason is the one from stop 4: you cannot guarantee that the same agent run twice does the same thing, and you cannot guarantee it will not repeat a step after a timeout. Rather than telling it in the prompt not to double-charge — stop 6 covered how weak that is — make double-charging impossible.
Can the model decide whether to retry?
It can take part, and it cannot be the only thing deciding. The model sees only what you wrote into the tool_result (stop 2), so whether the work happened has to be put there by your code before it can judge correctly. And even when it judges correctly it is only likely to — stop 4 still applies. The workable split is that safety comes from the structure of the tool, and the model chooses among options that are already safe.
Either give an agent tools that are safe to call twice, or give them a key that makes the second call do nothing. Retries and backoff are the easy part; the hard part is knowing which actions can be repeated without consequence.
It usually goes like this: one task gives an answer you do not like, you add a sentence to the system prompt, you run that task again, it is better, and the change stays. The only problem with the procedure is that the change is global and the check was local. That sentence now appears in every round of every task, and you looked at one of them.
Worse, a regression rarely looks like one. It does not raise an error and it is not obvious nonsense; it looks like a slightly shorter answer, or one item where there used to be four — and having just changed the prompt yourself, you are primed to read that as concision.
No framework, no grading model, no statistical significance. Two things: a saved list of tasks, and for each one a pass condition you can decide on the spot. "Answers well" is not a condition. "Names next, react and react-dom and invents nothing" is — the difference being that a second person applying the second one lands in the same place.
Ten tasks is enough to start, and judging them by hand is fine. Two things to watch when choosing them: keep the problems you have already fixed in the list, because they are the half that catches regressions; and include two or three tasks that should be refused — a file that does not exist, something the toolset cannot do — because the common failure is not silence, it is fluency.
What if the same task gives different results on different runs?
It will, and that is itself worth knowing. Run each task three times and record how many of the three passed, rather than pass or fail. A task that used to pass three times out of three and now passes two would be invisible if you ran it once. Consistency is part of quality.
The table above is a ten-item list. Switch to v2 — those two sentences were added to fix row 4, and they do fix row 4, and rows 7 and 10 as well. Looking only at those three, it is a clear success.
But rows 2 and 9 got worse, both for the same reason: the requirement to cite a file and a line for every claim made the model unwilling to state conclusions it could not pin down item by item, and a complete list became an incomplete one. The net result is 5 to 6: one point for the change. Whether that point is worth it is a fair discussion to have. Without this table you do not get to have it — you believe you gained three. And each task in the table ran once: a net gain of one may itself be sampling noise (stop 4), which is the reason, given above, for running each task three times.
Changing a prompt is not a local edit, it is a global one: a sentence added for one task applies to every task. So trying the task you were annoyed by is never enough — you have to run all ten. The difference between ten and one is the difference between knowing and assuming.
What it is gave you the shape: an and a loop, and then you wrote one. What the model is like covers two properties of the model itself: it varies (stop 4) and it invents (stop 5). The text you write covers the two kinds of text you write and what each can change. What it costs is money, a ceiling, and one way of pushing both somewhere else. When it goes wrong is everything off-plan: text that lies to it, actions that cannot be undone, calls that fail. How you know is how you tell whether a change helped (stop 13).
These are not parallel facts; they share one structure. Everything the model knows about the task is in the request — the array, the system prompt and the tool descriptions — and everything it can do is in that tool list. Faced with behaviour you have not seen before, asking those two questions usually locates it.
The four areas above are what this course deliberately does not cover. Each says why, because the reason is usually more useful than the material: it tells you when to go and look, and when you do not need to yet.
After fourteen stops what you have is a mental model you can take apart: one array, one loop, some text you wrote, some tools you supplied, and the ways each of them breaks. Its value is not that it is complete but that when you meet something you have not seen, you know which layer to open.