Stop 7 · How to describe a tool
A tool's description is part of the prompt, not documentation for a colleague. Three things people leave out, and what each one costs.
Every model behaviour on this stop is a written illustration, not a live API call. The numbers are computed from the assumptions stated below.
What a tool definition looks like
The tools parameter you send to the is a list, and each entry has three parts: a name (what the model writes when it asks for it), a description (a passage of ordinary language), and an input_schema (the shape of the arguments, as JSON Schema). The name and the schema have two readers: the model asks in that format, and the name is also one of its clues for choosing; your code dispatches and validates against them.
The description is different. It has no required format, takes part in no validation, and changes no line of your program. Its one reader is the model, and its only effect is to change what the model picks. In other words, it is a prompt. A sentence added here does the same kind of work as a sentence added to the system prompt — while a comment above the function body is never seen by the model.
Beginner question: How long should a description be?
It costs s like any other prompt, and it is resent on every round (stop 8), so longer is not better. A workable bar: write until someone who has never seen the project could decide correctly, from that passage alone, whether to use it. Two or three sentences usually do it — one for when to use it, one for the boundary, one for what comes back and how much.
what this task exposes: The description never says what the tool does not do
- the model asks for
- list_dir(".")
- the tool returns
- app/ docs/ lib/ package.json README.md tsconfig.json verify.mjs
- final answer
- This repository has 7 files.
Wrong, and wrong for an understandable reason. "Lists a directory" never said it lists only one level, so the model read the output as a complete inventory. The dozens of files inside app/ and lib/ were never seen, and the answer is stated with full confidence.
- the model asks for
- list_dir(".") → list_dir("app") → list_dir("lib") → …
- the tool returns
- Four calls, one level at a time: 7 at the root, 11 in app/, 12 in lib/, 2 in docs/.
- final answer
- 29 files including subdirectories, plus 3 directories.
Three more rounds, and a correct answer. The only change is one sentence: "does not recurse" and "at most 200 entries" are now written down. When describing a tool, what it does not do matters as much as what it does. A model does not guess at boundaries; it assumes a limit that is not written does not exist.
The three things people leave out
The three tasks above correspond to three omissions. What they have in common: every mistake in the left-hand column traces back to the description. The model reasons from an incomplete description, and the reasoning is not far-fetched. (In the second task its claim to have made the change is invented — but nothing in the description warned it that the toolset only reads.) That is also why a stronger model usually does not fix this class of problem: what is missing is not reasoning, it is information.
There is a practical test that requires running nothing: copy the descriptions out of your tool list, on their own, with nothing else, and ask yourself whether you could answer, for each tool, when to use it, what it does not do, what comes back, and how much. Wherever you cannot, the model cannot either — and it will not tell you that it is guessing.
Beginner question: What if two tools overlap?
First ask whether they should be one tool. If two are genuinely needed, write the boundary into both descriptions as a rule that can be followed, and have them name each other: "for questions about this project use search_files, not search_web", and "search_web is only for information that could not be in this project". Describe each in isolation and the model re-guesses the boundary every round — and it will not necessarily guess the same way twice.
Describing a tool means writing down four things: when to use it, what it does not do, what comes back, and how much. Leave one out and the model will assume on your behalf — and its assumption is always that a limit you did not write down does not exist.
A look back: the text you write
Two questions, both taking a passage someone would really write and asking what it actually guarantees.
Nothing is scored here. Every question can be answered again, and a wrong answer names the idea it came from.