Stop 9 · When it will not fit
The context window is a hard ceiling. There are four ways to deal with it — three once it is full, and one that keeps material out in the first place — and all four lose something.
Every model behaviour on this stop is a written illustration, not a live API call. The numbers are computed from the assumptions stated below.
First, what a token actually is
A model does not measure text in characters or in words but in s. Split a sentence the way a model actually sees it and you get something like: "Find / out / why / the / order / tot / als / are / wrong" — common words are one piece, unusual ones get broken up, and punctuation and spaces take up room too. In English a token is roughly four characters; in Chinese the ratio depends on the tokenizer — a common word of one or two characters may be a single token, while a rare character can take several.
Why it matters: there is a ceiling on how many tokens a model can read in one request, and that ceiling is the . It is not a soft limit that makes things slow. It is a wall — one token over and the request is not accepted at all. The 12 MB log that failed at stop 2 hit this wall.
Beginner question: The windows are large. Does anyone actually run out?
In ordinary chat, almost never. In a tool-using agent, easily: a directory listing, a few files, a test run, a stack trace, and a dozen rounds puts you in the tens of thousands. Add someone pasting a PDF or a whole schema into the first message and it goes fast. And remember stop 8: staying under the ceiling is not the same as being cheap. Everything you keep in the window is paid for again every round.
- systemsystem promptYou are a database maintenance assistant; you can read files and run queries.120
- userdroppedthe tasktaskThe order totals have been wrong since last week's migration. Find out why.40
- assistantdroppedlet me read the configtool_use: read_file("config.json")30
- userdroppedtool_result: config.jsonConnection settings, timezone UTC+0, decimal places 2.180
- assistantnow the migrationtool_use: read_file("migrations/0042.sql")30
- usertool_result: 0042.sqlChanged total from DECIMAL(10,2) to FLOAT.320
- assistantand the error logtool_use: read_file("logs/orders.log")30
- usertool_result: orders.logMany totals like 0.30000000000000004 instead of 0.30.400
- systemsystem promptYou are a database maintenance assistant; you can read files and run queries.120
- assistantnow the migrationtool_use: read_file("migrations/0042.sql")30
- usertool_result: 0042.sqlChanged total from DECIMAL(10,2) to FLOAT.320
- assistantand the error logtool_use: read_file("logs/orders.log")30
- usertool_result: orders.logMany totals like 0.30000000000000004 instead of 0.30.400
Note: the task itself was dropped. The model can no longer read the question it is answering.
Three ways past it, all of them lossy
The conversation above has run three rounds and comes to 1,150 tokens, against a window of 1,000 in this example. The next round cannot be sent. There are exactly three things to do about it; try each with the buttons above:
Refusing is the most honest and the least useful: tell the user it does not fit and stop. In many products that is the right call — better than an answer built on a mutilated history. Truncating is the most common and the easiest way to fail quietly: drop from the oldest end. Try it and watch what goes first — the task itself, after which the model is answering a question it can no longer read. What is left also starts with an assistant message, which many APIs refuse; real truncation has to keep a user message first. Summarising costs the most and helps the most: spend an extra model call folding the early stretch into a short passage and put that back. It keeps what the job was, at the price of detail that is gone for good — the summary no longer says that config.json sets two decimal places, or that total used to be DECIMAL(10,2), and nothing can recover either.
Beginner question: Why not just drop the big ones in the middle?
You can, and in practice people do, but be careful: messages in the array are not independent. An assistant tool_use and the tool_result after it are a pair. Drop the result and keep the request, and the model sees itself asking for something that never came back. Keep the result and drop the request, and an answer appears that nobody asked for. Drop them in pairs, and make sure what is left still reads as a coherent history.
Read the 30 files under lib/ and tell me which ones touch authentication.
Four files touch authentication: lib/auth/session.ts (reads and writes the session cookie), lib/auth/verify.ts (validates the JWT), lib/api/client.ts (attaches the Authorization header) and lib/middleware.ts (redirects when signed out). The other 26 are unrelated.
| The caller sees | Actually | |
|---|---|---|
| Rounds | 1 (asked, answered) | 31, inside the subagent |
| Files read | not visible | 30, four of them truncated |
| Tokens spent | about 400 (question and answer) | about 128,000 |
| Growth of the parent array | 2 messages, about 400 tokens | the same; none of the subagent's 62 messages came across |
| What the conclusion rests on | a 300-token passage | the contents of 30 files |
There is in fact a fifth: lib/cache.ts builds cache keys from the user id, so the cache is not partitioned by session. The subagent read that file truncated, the truncated part is exactly those lines — and its summary does not mention that anything was cut.
A fourth way: keep it out of this conversation
The three ways above all trade inside one conversation: what stays and what goes. There is a fourth, which is that some material should never enter this at all. "Read the 30 files under lib/ and find the ones that touch authentication" is the shape: thirty files come in and are resent every round afterwards (stop 8), and what you wanted was a list of four or five names.
The move is to start a second agent with its own array, its own loop and its own budget. It reads the thirty files across thirty-one rounds and hands back only the conclusion. On the caller's side the whole episode is one tool call: one message out, one message back. The two columns above are two ledgers for the same event — the caller's array grew by about 400 tokens while the episode cost a hundred and twenty-eight thousand. What was saved is not those hundred and twenty-eight thousand but the fact that they are not resent on every later round.
The price is in the last two rows, and it is a different kind of loss from the first three. Truncation loses content, and you know what you lost. Handing work off loses the ability to check: the caller holds a 300-token passage and not the thirty files, so it has no way to verify that passage. This summary did miss a file — the subagent read lib/cache.ts truncated and never mentioned it — and what reached the caller was a list that reads as complete.
This is the shape from stop 2 again, one level up: there a tool failed to report a failure, and here an entire agent failed to report a truncation. Every boundary you add is one more place information can quietly disappear, and the outer layer never sees inside the inner one. The workable middle is to have the subagent return a few lines of metadata alongside its conclusion: how many files it read, whether anything was truncated, whether a tool failed, and where it was unsure. Those lines cost almost nothing and turn a silent omission into a visible doubt, which is something you can act on.
Beginner question: When is handing work off worth it?
Three questions: does it read a lot, is the conclusion short, and would a mistake be visible? Reading thirty files and returning a four-line list answers yes to all three, and is worth it. Editing three files fails the third — a bad edit announces nothing, and the caller holds no original to compare against. Put the other way round: when a mistake would fail quietly, do not put a boundary in front of it.
A full context is not a problem intelligence can solve. It is a question your code has to answer: what gets thrown away. Refuse, truncate, summarise, or hand the work to another agent — each loses something different, and where the first three lose content the fourth loses the ability to check. Declining to choose is choosing the worst one, where the request is simply refused.
A look back: what it costs
Three questions. All arithmetic, and none of them a number the text handed you.
Nothing is scored here. Every question can be answered again, and a wrong answer names the idea it came from.