Cost control for agent systems: where the tokens actually go

Where the tokens actually go in an agent system

The obvious move is to compare prices per million tokens and pick a cheaper model. That changes the price of a token. It does not change how many tokens a task needs, and in an agent loop the count is what moves the bill. Almost none of that count is the words a person typed. Every price and limit in this article is quoted from the vendor page named beside it, read on 19 September 2026. The only measurement of our own is two payload sizes from our own server. There is no benchmark here, no customer result and no adoption number, because we have not run those.

Here is the shape of a single agent turn. The host sends the system prompt, the full schema of every tool the agent can call, the whole conversation so far, and the result of the last tool call. The model writes a short reply and maybe one more tool call. Then the loop runs again, and everything above the new line is paid for a second time. A ten step task does not cost ten times one message. It costs the sum of ten growing prompts.

That is why the six line items below are worth separating. Each one grows for a different reason, each one is measured a different way, and the fixes are not interchangeable. The order here is the order to read them in, not the order of size, because the size depends on your system.

One number of our own, measured on our own server by weighing the response payloads. On Growth OS, which compiles 48 agents, the pack our talk tool returns for a single conversation turn is about 2 KB. A full read of the agent roster is about 110 KB. Same question, two ways of answering it, and the gap is the bytes that reach the model. That is the difference between answering from a compact pack and letting the host read everything first.

• System prompt: fixed size, paid on every request, every agent, every turn.
• Tool schemas: grow with the number of tools, paid on every request whether or not a tool is used.
• Tool results: the most variable of the six. One roster read can outweigh the whole conversation.
• Retries and failed calls: pay the full prompt again for zero progress.
• Fan-out: each sub-agent re-pays its own system prompt and schemas before it does any work.
• History: the conversation is resent on every turn, so a long task pays for its own past repeatedly.

The system prompt and the tool schemas are a subscription

A system prompt is written once and billed forever. So is every tool schema. If your agent has forty tools, the name, description and full input schema of all forty are in front of the model on every single request, including the ones where the model just says yes.

This line item is invisible. It never appears in a log, and nobody watches a tool schema scroll past. Measure it once and the 300 word tool description stops looking free. The Claude API has a token counting endpoint that takes the same body as a real request, tools included, and returns the input token count without running the model. Send your request body twice, once with the tools array and once without it. The difference is the schema tax you pay on every call.

Two traps here that come from the vendor docs. First, Anthropic's token counting page at platform.claude.com/docs/en/build-with-claude/token-counting says the count is an estimate, and that the endpoint rejects some inputs the Messages API accepts, including server tools and the MCP connector, so for those you have to read the usage object on a real response instead. Second, the same page says Claude 4.7 and later models use a newer tokenizer, where the same input text produces roughly 30 percent more tokens than on earlier models, and tells you to recount against the model you plan to use. A budget carried over from an older model can be off by roughly that much before you start.

The fix is not to write shorter prompts. The fix is caching. Anthropic's pricing page at platform.claude.com/docs/en/about-claude/pricing lists a cache hit at 10 percent of the base input price, a five minute cache write at 1.25 times the base price, and a one hour write at twice the base price. The same page states the break-even: a five minute cache pays for itself after one read, a one hour cache after two. It lists Claude Opus 5 at 5 dollars per million input tokens, 6.25 for a five minute write, and 0.50 per million on a cache hit.

Caching works on prefixes, so the order of your request decides whether it works at all. Anything that changes invalidates everything after it. A timestamp in the system prompt, a tool list that is built by iterating a hash map in a different order each run, or a user name injected at the top, and your cache read rate is zero while you keep paying write prices.

• Put the stable content first: system prompt, then tools, then history, then the volatile part of the turn.
• Freeze the tool order. Sort it. A set iterated in random order is a cache miss you will never notice.
• Watch cache_read_input_tokens in the usage object. Zero across repeated calls means something upstream is changing.
• Delete tools the agent never calls. A schema you do not use is rent you pay on every request.

Tool results: the line item that keeps getting paid

A tool result goes into the conversation and stays there. It is paid on the turn it arrives, and again on every turn after that, until something removes it. One careless tool can therefore outweigh every other line item in the loop.

The usual shape of the mistake is a list endpoint. An agent asks what is available, and the tool answers with everything: every record, every field, every id. That tool is simple to write, and its result is then carried by every turn after it. The two payload sizes we weighed on our own server are the example: about 2 KB for a pack that answers the turn, against about 110 KB for the full 48 agent roster. Same system, two different tools.

Two hosts cap this by default, which is itself informative. Anthropic's Claude Code MCP page at code.claude.com/docs/en/mcp documents a warning threshold at 10,000 tokens of tool output and a default limit of 25,000, with results above the limit written to a file instead of the conversation, and an environment variable to raise the ceiling. It also documents a per tool annotation a server can send in its tools list to raise its own result size, up to a maximum of 500,000 characters. OpenAI's MCP page at learn.chatgpt.com/docs/extend/mcp documents a per tool output token limit in the Codex config, applied before what it calls the standard 20 percent serialization allowance.

Read those caps as a warning rather than a solution. A truncated tool result is a tool result the model cannot use. The agent will call the tool again with different arguments, which costs a full turn, or it will answer from the part it saw, which is worse. Fix the tool, and use the cap as a tripwire that tells you when a tool has regressed.

The fix that works is a pack: the tool decides what matters for this turn and returns that, with a handle to fetch the rest if it is needed. This moves work from the model to your server, which is the trade. Your server now has to be smart about what to include, and when it guesses wrong the agent needs a second call. Whether the trade pays depends on how often the guess is wrong. One extra call now and then beats a roster read on every turn. A pack that misses half the time does not.

• Cap output at the source, in the tool, not at the host. The host cap truncates. The tool can summarize.
• Return identifiers plus a fetch tool, rather than whole records the model will mostly ignore.
• Strip fields no agent reads: timestamps, internal flags, empty arrays, repeated parent objects.
• Log the byte size of every tool result. The worst one is not the one you would guess.

Retries and failed tool calls: the cost that never shows up in the UI

A failed tool call costs exactly as much as a successful one. The model still read the prompt, the tools and the history to produce the call. Then it reads the error and tries again, which costs all of that a second time. Three failures in a row is four turns of input tokens for one step of progress.

Timeouts are the expensive variant, because the host waits and then bills. OpenAI's MCP page at learn.chatgpt.com/docs/extend/mcp documents a default tool timeout of 60 seconds per call and a default startup timeout of 10 seconds per server, both configurable per server. A tool that sits at 59 seconds and then fails pays full price for that turn, produces nothing, and then gets retried.

The cheapest fix is a schema the model cannot get wrong. Enum the values it must choose between rather than describing them in prose. Mark the required fields. Give one example in the description, not four. When a tool fails twice with the same error, read the argument shape before you touch the prompt. An argument shape failure is fixed in the schema.

The second fix is making a retry safe. If a call can be repeated without doing the work twice, the agent can retry without you worrying, and you can fail fast instead of holding the connection open. If a call cannot be repeated safely, it should not be retried automatically at all, and it should be behind a human approval before it runs.

• Count turns per completed task, not tokens per request. Retries show up there and nowhere else.
• Log the error string of every failed tool call, then group them. A short list of distinct strings is normal, and each one is a fix.
• Fail fast on tools that cannot succeed. A 60 second timeout on a dead endpoint is a full-price turn.
• Return a usable error: what was wrong and what to send instead. An error the model can act on prevents a third call.

Multi-agent fan-out multiplies the prefix, not just the work

Splitting a job across several agents is a good idea for quality and a dangerous one for cost. Each agent starts with its own system prompt and its own tool schemas. A fan-out to six sub-agents pays six prefixes before a single useful token is produced, and their answers all come back into the parent conversation, where they are paid for again on every later turn.

The arithmetic to do before building one: prefix size times the number of agents, plus the tokens each one produces, plus the cost of their combined output sitting in the parent context for the rest of the task. If the prefix is large and the work per agent is small, a fan-out is a way to spend more money for the same answer.

Caching does not soften this, because a cache is tied to one exact prefix and one model. Six agents with six different system prompts are six separate caches, each one cold on the first call. Six agents that share one prompt and differ only in their instructions can share the prefix, which is a design decision you make once and keep.

The rule we use: fan out when each branch reads a lot and returns a little. Research across five sources fits that shape, because the reading stays inside the sub-agent and only the conclusion comes back. Splitting one short task across five agents does not fit it, and the bill says so.

Long conversations re-read themselves

The API is stateless. Every turn resends the whole conversation. That means the cost of turn twenty includes everything from turns one to nineteen, and the total cost of a task grows faster than the number of steps. A demo of three turns and an hour of real work have very different shapes for this reason.

The same effect hits files. An agent that reads a large file, works for ten turns, then reads the same file again because it does not trust its own summary has paid for that file twice in full, plus the cost of carrying the first copy through all ten turns in between.

Three mechanisms bound it, and they do different things. Caching makes the resent history cheap rather than free. Dropping old tool results removes them from the conversation. Summarising old turns replaces them with shorter text. Hosts and APIs give these different names, so check what yours calls them. The last two lose information, so they are decisions about what the agent may forget, not free wins.

Cache lifetime matters here in a way that is easy to miss. With a five minute cache, a developer who reads the last answer, thinks for ten minutes, and then replies has let the cache expire. The next turn pays write prices again on the whole history. If your usage pattern is bursty, the longer cache lifetime that Anthropic's pricing page lists at twice the base write price can be the cheaper option, and the break-even it states is two reads.

• Measure the token count of turn one and turn twenty of the same task. The ratio tells you how bad the growth is.
• Cache the history prefix on long tasks. It is the difference between paying full input price and 10 percent of it.
• Have the agent write its conclusions into a compact note it can reread, rather than rereading the source.
• Decide what the agent may forget before you turn on anything that edits or summarizes context.

How to measure all six in an afternoon

This is the part that pays. Do it once on a real task, in this order, and you will know which of the six line items is yours. What you need is a token counter, the usage object from the responses you already get, and somewhere to write six numbers down.

Step 1. Baseline the prefix. Take a real request body and send it to the token counting endpoint: curl -s https://api.anthropic.com/v1/messages/count_tokens -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "content-type: application/json" -d @request.json. What you see: a JSON object with an input_tokens field. Write that number down.

Step 2. Split the prefix. Copy request.json to no-tools.json, delete the tools array, and count it again. The difference between the two counts is the tool schema cost you pay on every request, forever. Divide it by the number of tools to find out which ones are worth trimming.

Step 3. Weigh the tool results. Run the task once with logging on every tool return, and record the byte size of each result. Sort that list. The top entry is usually the thing to fix first, and it is usually a list call.

Step 4. Read the usage object from a real run. On the Claude API each response reports input_tokens, output_tokens, cache_creation_input_tokens and cache_read_input_tokens. Sum them per task rather than per request. If cache_read_input_tokens is zero across repeated runs of the same task, your cache is not working, and that single fact is often worth more than every other change on this page.

Step 5. Count turns per completed task. Not requests, not tokens: completed tasks and the turns they took. Then look at how many of those turns produced a failed tool call. That is your retry tax, expressed in a unit you can act on.

Step 6. Put the six numbers next to each other and fix the largest one. Then measure again, because the second largest is rarely what it was before you started.

The honest limit of this procedure: a token count is not a bill. Cached reads, batch discounts and different models are all priced differently, so the counts tell you where the tokens are, and the pricing page tells you what they cost. Keep the two separate in your head or you will optimize the wrong number.

The interventions that work, in order of payoff

Do these in order. Each one is cheaper to try than the one after it, and the early ones cost you nothing in quality, which is the part that matters when someone asks why the answers got worse.

First, cache the stable prefix. It is a configuration change, it does not alter a single word of output, and on repeated work it turns the largest fixed cost into a tenth of itself. Verify it with the usage object rather than assuming it worked.

Second, fix the loudest tool. One tool returning a full roster on every call can outweigh every other line item combined. Return a pack sized to the question, keep a fetch tool for the rest, and put a size check in your own code so a regression is caught before the invoice catches it.

Third, cut the schema tax. Delete unused tools, shorten descriptions to the sentence the model actually needs, and move the long explanation into the server where it costs nothing per request.

Fourth, batch what is not interactive. Anthropic's pricing page states the Batch API is a 50 percent discount on both input and output tokens, and that it combines with caching. Overnight jobs, backfills and bulk classification belong there. Anything with a person waiting does not.

Fifth, and only now, touch the model and the reasoning effort. This is the first lever that trades quality for money, which is why it is fifth and not first. Judge it on cost per completed task, because a cheaper model that needs three attempts is not cheaper.

What does not work, and what this does not solve

Swapping to a cheaper model as the first move is the most common mistake. It changes the price of every token while leaving the number of tokens untouched, and if the cheaper model needs more turns to finish the same task, the bill goes up while the dashboard says the rate went down.

Compressing prompts by hand is the second. Rewriting a system prompt to save 200 tokens is a day of work that caching would have made irrelevant. Do the caching first, then decide whether the prompt is still worth editing.

Counting tokens instead of tasks is the third. A dashboard of tokens per day cannot tell you whether the work got done. Cost per completed task is the only number that survives a change in model, prompt or workflow, and it is the only one worth putting in front of a finance team.

What none of this solves: if your agent is slow because a person has to approve every step, tokens are not your bottleneck and no amount of profiling will help. The same is true when the real cost is an engineer rerunning a broken pipeline. Measure where the hours go before you optimize where the tokens go.

A second limit worth naming: every number on this page that is not ours comes from a vendor pricing or docs page, and those change. The multipliers, the caps and the discounts have all moved before. The method holds; recheck the numbers against the pages in the sources before you build a budget on them.

Where Agentik fits, in one paragraph

Agentik {OS} installs on the host you already pay for, and the host runs the model and pays the tokens. We never buy your tokens, so a cheaper pack is not a cheaper invoice for us, it is a cheaper invoice for you. That is the reason the talk tool exists: it answers a normal conversation turn and returns a pack of about 2 KB on Growth OS instead of the roughly 110 KB the full 48 agent roster costs to read. Work that publishes, sends, spends or resets waits for a human approval, which also caps the cost of a wrong turn.

The honest limit of our own design: a pack is a guess about what this turn needs. When the guess is wrong the agent asks again, and that second call costs a turn. We think that trade is right for conversation and wrong for a task that genuinely needs the whole roster, and when you need the whole thing you should read the whole thing.