Stop 6 · The text outside the array
The system prompt is not in messages, and it goes out on every single round. It can change behaviour, and changing behaviour is all it can do.
Every model behaviour on this stop is a written illustration, not a live API call. The numbers are computed from the assumptions stated below.
Task: Summarise last week's error log and send it to me.
You are an operations assistant. You can use tools to help the user with things on the server. The current user is ada@example.com.
read_filerun_commandsend_email
- 1read the logread_file("logs/errors.log")
- 2send itsend_email(to: "ada@example.com", body: "312 errors last week…")
- 3outcomeThe mail went out. Send it to me was taken literally, which is not an unreasonable reading: it is what you wrote.
Nothing here was done wrongly. The task says send it to me, the tool list contains send_email, and the system prompt never said otherwise and even gives the user's address. The model joined those facts and reached a perfectly natural conclusion.
Where it sits in the request
The code panel at stop 2 has this line: system: "You are a local file assistant…". It is a parameter alongside messages, not an element of the . The distinction is deliberate: the array grows with the conversation, gets truncated, gets summarised (stop 9), while the system prompt is the passage you supply again, unchanged, on every round. It cannot be squeezed out by a long conversation.
Supplied again, unchanged, on every round has two consequences. The good one is stability: however long the conversation runs, that passage is still there and still first. The other one is the bill: it is charged again every round.
You pay for it once per round
Apply the rule seen at stop 2 — everything is resent every round — to the system prompt: a 900- passage in a forty-round run goes out forty times. You wrote it once and bought it forty times — the figure above is computed with the same cost model stop 8 uses.
This is not an argument for short prompts. Nine hundred well-spent tokens usually save far more than they cost. What is worth remembering is the other thing: every sentence in a system prompt is billed per round, so a paragraph added just in case costs you not once but every time. And a related point: it already sits ahead of the conversation, so it earns the cache discount from stop 8 as long as it stays byte-identical — drop a current timestamp into it and the whole prefix stops matching.
Beginner question: Do the long you must follow these rules lists work?
They work, with returns that fall off quickly and failures that are hard to see. For a rule to take effect it has to win, on every round, against every opposing pressure in the context at that moment — the wording of the task, the contents of a tool result, and the sampling from stop 4. Ten rules that matter usually outperform forty. And the one or two that matter most are better not implemented as rules at all, which is the next section.
Telling it is the weakest control you have
The three conditions above run the same task. In the first, nothing was said and the mail went out — and no step was taken wrongly: the task says send it to me and the tool list contains send_email. In the second, a sentence was added to the system prompt and nothing went out this time. In the third, every tool that could send anything — send_email, and run_command, which runs anything — was removed from the list and nothing could go out.
The second and third produce the same outcome and are not the same kind of thing. The second is a passage of text sitting alongside other passages of text: it has to win again on every round against the task's wording, the tool results, and the sampling from stop 4. It wins nearly always, and nearly always is not always. The third enters no contest at all — a tool that is not in the list cannot be requested, regardless of what the model was thinking or what it had just read.
stop 10 takes this to its limit: if a piece of outside content can address the model, then I told it in the system prompt becomes a contest with an attacker over whose text is more persuasive. You will not necessarily lose that contest, and you will not necessarily win it, and you do not know in advance. So the order of the questions should be: first ask whether the thing can simply be made impossible, in which case no instruction is needed; only when it has to remain possible do you fall back to moving a tendency with a prompt.
A system prompt moves a tendency; the tool list decides what is possible. "I told it to be careful" is an order of magnitude weaker than "I did not give it that tool" — the first has to hold again on every round and against every piece of outside content, while the second, once every route is closed, holds once and holds for good.