Skip to content
RingMod

ring·mod: the synth circuit that multiplies two signals into one neither could make alone. Your platform × AI, under control.

← Notes
Field note

The meter said $17.66. The work cost $4.46.

We audited every cost meter in our agent toolchain against 224 graded runs. Four meters gave four different numbers: a 4x display error on proxied runs, a sticker price that was never charged, a live usage feed that hides cache hits, and reasoning tokens billed with no visible text.

Last updated

When we ran GPT-5.6 Sol through our agent battery inside Claude Code two weeks ago, the harness’s cost display priced the full eight-task pass at $17.66. The proxy sidecar we had bolted on, which logs every request’s raw usage and prices it against OpenAI’s published rates, said $4.46. Same runs, same tokens, one number 4x the other. We published the $4.46 and moved on. This week we went back through every cost meter in the toolchain to find out how many others were wrong, and by how much.

The dataset is 224 graded runs from three public bouts, all in the same Claude Code harness: four Claude models (Haiku 4.5 through Fable 5), GPT-5.6 Sol through a LiteLLM translation proxy, Kimi K3 through Moonshot’s Anthropic-compatible endpoint, and GLM-5.2 through Z.ai’s. Fable and Opus each appear in two bouts, which makes nine arms from seven configurations. Every run is priced regardless of grade: 222 of 224 passed their deterministic graders, and the two that failed (both Haiku, one task) stay in the totals, because a meter bills failed work too. We recomputed every bill from per-run usage records at list prices, split into four token classes: uncached input, cache writes, cache reads, and output. The design and hypotheses were committed before any aggregate was computed. One prediction missed; it’s below with the meters.

One more disclosure before the numbers. This note, like the series, is drafted by Fable 5, which is one of the nine arms being priced, and the four Claude arms run on Anthropic’s home harness. Every dollar here comes from a deterministic script over committed usage records, not from a model’s judgment; there is no LLM judge in this note, and the founder reviews everything before it publishes.

Four meters, four different numbers

The display price for proxied models ran 2x to 6.3x hot. Claude Code prices what its Anthropic-format accounting can see. The translation layer between it and OpenAI drops the cached_tokens field (LiteLLM upstream issues 27763 and 9812), so every cached read gets billed at the full input rate in the display. Here is one request from the sidecar log (01-bugfix/gpt-5.6-sol/run-1/proxy_usage.jsonl), trimmed:

{"model": "openai/gpt-5.6-sol", "response_cost": 0.0173885,
 "raw_usage": {"prompt_tokens": 22822, "completion_tokens": 152,
               "prompt_tokens_details": {"cached_tokens": 22507}}}

22,507 of 22,822 prompt tokens were cache hits at a tenth of the input price. The display saw 22,822 uncached tokens. Across all 24 Sol runs that compounds to $17.66 displayed against $4.46 recomputed per pass (bouts/2026-07-17-fable-sol-kimi, per-run ratios 2.0x to 6.3x). The GLM-5.2 arm had the same class of error at 3.21x: $0.343 displayed against $0.107 per run. Our published notes used the sidecar figures both times, which is the only reason the series’ cost claims survive this audit.

The display priced Sonnet 5 against a sticker that was never charged. Anthropic billed Sonnet 5 at $2/$10 per MTok, scheduled to rise to $3/$15 on September 1, and the CLI’s cost field for our July 7 ladder bout reproduces exactly at $3/$15. That rise will not happen. On August 10 Anthropic cancelled it and made $2/$10 the standard price. Every class scales by the same factor, so the display ran exactly 1.5x the invoice: $0.2152 shown, $0.1434 billed, per run. The ladder’s capability findings don’t move; the dollars we printed were the conservative ones.

Kimi’s live usage feed is unusable for cost. Moonshot’s Anthropic-compatible endpoint reports, mid-run, the whole prompt as input_tokens with the cache fields zeroed (01-bugfix/kimi-k3/run-1, transcript record vs final reconciliation):

{"input_tokens": 22998, "cache_read_input_tokens": 0, "output_tokens": 0}

The end-of-run reconciliation for the same run says 25,551 uncached in, 123,136 cache reads, 976 out. Anything watching the stream to enforce a budget would think this arm cost 4x what it did. Only the final aggregate is real.

Reasoning tokens are billed output you cannot read. We pre-registered a hypothesis here and missed. Expecting Kimi K3 to be the verbose one, we predicted it would still spend fewer output dollars than Sol because its output price is half. Wrong premise: Sol billed 7,786 output tokens per run against Kimi’s 3,112, a 2.5x gap. OpenAI’s own usage detail says where they went: 5,150 of Sol’s billed output tokens per run are reasoning (completion_tokens_details.reasoning_tokens, summed by the committed script), 66 percent of its output bill, with no visible text. Sol’s visible output, 2,637 tokens a run, is smaller than Kimi’s entire billed total. Kimi prints more text and spends $0.047 a run on output; Sol prints less and spends $0.234, most of it on tokens nobody will ever read.

Why the errors get this big

Meters break where the money is, and for agents the money is not where the tokens are.

Share of tokens versus share of dollars per run, by token class, for four model arms Uncached input + cache writes Cache reads Output Sonnet 5 tokens 94% dollars 28% 41% 31% Opus 4.8 tokens 8.2% 89% dollars 30% 28% 42% GPT-5.6 Sol tokens 6.7% 91% dollars 25% 34% 42% GLM-5.2 tokens 6.3% 91% dollars 20% 55% 25% List prices. 30 runs each (Claude arms), 24 (Sol, GLM). Same six-to-eight task battery, Claude Code harness.

Across all nine arms, cache reads are 87 to 94 percent of processed tokens but only 26 to 55 percent of dollars. Output is 1.1 to 3.3 percent of tokens and 25 to 48 percent of dollars. The cache shares we published in earlier notes (93.4 percent for GLM, 93 for Kimi) were input-side shares; today’s are all-token, which output dilutes. The two classes a meter is most likely to mishandle, discounted reads and premium output, are between them most of the bill. A translation layer that drops one field, the cached-token count, moves the number 4x because that field covers 90 percent of the traffic.

The flip side: the effective input-side rate (all input-side dollars over all input-side tokens) came out between 15.8 and 23.8 percent of the sticker input price in every arm, all four vendors. That band is narrow, and it’s mostly arithmetic: when ~90 percent of your context arrives at a 10 to 19 percent read discount, the blend has nowhere else to land. Sticker prices still rank models correctly. GLM-5.2 has the weakest read discount of the four vendors (cached reads at 18.6 percent of its input price against 10 percent for the others) and was still the cheapest non-Claude arm at $0.107 a run. The cheapest arm overall, Haiku 4.5 at $0.058 a run, also has the lowest sticker. The ranking survives the blend. What sticker can’t tell you is the absolute bill, and the meters that should tell you were the four things above.

Scope, honestly: these are 30-to-120-second tasks in one harness at list prices, reconciled against vendor usage records rather than invoices. Longer sessions shift the mix toward reads even further. The Sol translation bug is version-pinned to our July proxy build and may be fixed upstream by the time you read this. The composition numbers are properties of this battery; the failure modes are properties of the tooling, and those travel.

What to log instead

Log raw usage per request, not the harness’s dollar rollup: uncached input, cache writes, cache reads, output, and for OpenAI-format vendors the prompt_tokens_details.cached_tokens field specifically, because that’s the one translations drop. Price it yourself against the vendor’s current sheet, and diff your number against the display’s number; the gap is your meter error, and ours ranged from 1.5x to 6.3x depending on which meter we asked. Put billed output tokens per run on the model scorecard next to visible output, because the difference is reasoning spend you can’t read. And treat any in-stream usage feed as a rumor until the end-of-run reconciliation confirms it.

All 224 runs, the per-request records, and the analysis script are in the public repo (analysis/2026-07-23-cache-economics).

The instrument we trusted least going in was the one we built ourselves, the sidecar. It was the only one that agreed with the arithmetic.

Questions this raises

Straight answers.

Why doesn't my agent harness's displayed cost match the provider's bill?
Usually because a translation layer between accounting formats drops a field, most often the cached-token count. In our 224-run audit, Claude Code's display ran 2.0x to 6.3x above the recomputed cost on GPT-5.6 runs routed through a LiteLLM proxy ($17.66 vs $4.46 per eight-task pass) and 3.2x high on GLM-5.2, because cache reads billed at a tenth of the input price were displayed at full price. The display also priced Sonnet 5 at $3/$15, a September increase Anthropic later cancelled, while the invoice ran at $2/$10: exactly 1.5x apart. Log raw per-request usage (uncached input, cache writes, cache reads, output) and price it against the vendor's current sheet yourself.
What actually drives an AI coding agent's API bill?
Not the headline input price. Across nine model arms and four vendors, cache reads were 87 to 94 percent of processed tokens but only 26 to 55 percent of dollars, and output was 1.1 to 3.3 percent of tokens but 25 to 48 percent of dollars. The effective input-side rate came out between 15.8 and 23.8 percent of the sticker input price in every arm, so agents pay roughly a fifth of sticker for context. Sticker prices still ranked the models correctly in our data; what they cannot tell you is the absolute bill.
Do reasoning models cost more than their visible output suggests?
Yes, and the gap is measurable. GPT-5.6 Sol billed 7,786 output tokens per run on our battery, but OpenAI's own usage detail shows 5,150 of those (66 percent) were reasoning tokens with no visible text. Its visible output, 2,637 tokens per run, was smaller than Kimi K3's entire billed output. Track billed output tokens next to visible output per run; the difference is reasoning spend you cannot read but pay full output rates for.

AI Cost Optimization

The writeup has a service behind it.

If this is your situation, the ai cost optimization is where it gets fixed — by the person who wrote this.

Request an audit