Stop 8 · Why it costs what it costs
The cost of a run does not grow with the number of rounds. It grows with the square of it. This stop draws the curve.
Every model behaviour on this stop is a written illustration, not a live API call. The numbers are computed from the assumptions stated below.
Back to the array
stop 2 showed this as a fact about how the API works: it is , so every round resends the whole from the beginning. Now treat it as arithmetic. Round 1 sends the system prompt, the tool list and your task. Round 2 sends all of that plus the model's reply plus the tool result. Round 3 sends all of that plus another round's worth.
The second step is the one that matters: you do not pay for the last round, you pay for the sum of all of them. In a run that adds six hundred s per round, round 40 sends about twenty-five thousand tokens — but rounds 1 through 40 together send over half a million. What grows linearly is the size of one round. What grows quadratically is the total. The bill is the total.
Beginner question: Why the square?
Line up the size of each round: a, a+g, a+2g, …, a+(n−1)g. That is an arithmetic series, and it sums to n·a + g·n(n−1)/2. The second term contains n², which dominates once n is large. In plain words: by round 10 you have sent the first message ten times, and by round 40 you have sent it forty times. The number of rounds is growing and so is the number of times the early content is re-bought. Two things growing together multiply.
The curve pulls away from the dashed line, which is what it would look like if every round cost what round 1 cost. By round 12 the real total is 2.5 times that straight line.
Assumptions: 1,200 tokens of system prompt and tool list, a 60-token task, 600 tokens added per round, 200 tokens generated per round; $3 per million input tokens, $15 per million output; a cache hit is charged at one tenth of the input price and a cache write at 1.25×.
Caching flattens the slope, it does not change the shape
Almost everything sent on a round is identical to the previous round; only a short piece at the end is new. Prompt caching exists for exactly this: if a round begins with the same content as the last one, that prefix can be re-read at a fraction of the normal price, and only the new part is paid for in full, plus a little more to write it into the cache. Turn the switch on above and watch the second curve.
Look carefully at the shape: the cached line still bends. The re-read prefix is billed at a tenth, but it is still billed once per round, so the curve is still quadratic with a much smaller coefficient. Under the assumptions on this page, forty rounds fall from $1.68 to $0.36 — about 4.6 times cheaper, not ten times, because output tokens are still charged in full and each round's new slice is charged at the cache-write rate, a quarter above it. Caching is a discount, not an exemption. An agent that runs five hundred rounds is expensive either way; with caching it is less expensive.
Beginner question: Is caching always worth switching on?
Almost always, with two conditions. First, what is cached has to be a prefix: insert one sentence at the start of the array and everything after it shifts, invalidating the whole cache. So a random greeting or the current timestamp does not belong in the system prompt. Second, writing to the cache costs a little extra, so a conversation that ends after a single round comes out about 14% more expensive with caching on. Drag the round count to 1 and the cached line sits above the other one; drag it to 2 and it is already below.
What goes in early costs the most
Turn on the second switch above: paste a twenty-thousand-token document into the first message — a specification, a log, a large JSON blob. It is pasted once, but it sits at the front of the array, so it appears in every request from then on. Over forty rounds you pay for that document forty times: six cents of tokens becomes $2.40, and the run goes from $1.68 to $4.08. In thirty-nine of those forty it may not have been needed at all.
The rule this gives you is concrete: keep bulk behind a tool and fetch it when the model actually asks, rather than loading it up front. The instinct to put everything that might be relevant into the context and let the model pick is, in a metered loop, the most expensive thing you can do.
Whatever you put into the on round 1, you buy again on every round after it. The thing to control is not how many rounds a run takes, but how much each round has to carry.