Skip to content
RingMod

ring·mod: the synth circuit that multiplies two signals into one neither could make alone. Your platform × AI, under control.

← Notes
Field note

Kimi K3 passed all 54 graded runs. The dial moved the thinking, not the bill.

54 graded Kimi K3 runs across three requested reasoning efforts in its own CLI: thinking volume swings 12.1x, the bill 1.4x, execution time 1.9x, every grade holds, and the blind rubric judge cannot separate the arms. On agentic work the bill is context traffic, and the effort dial does not govern it.

Last updated

Turn Kimi K3’s reasoning dial from low to max and it thinks twelve times harder: 2,205 characters of reasoning per battery pass against 26,665. The meter’s answer to all that thinking is 19 cents. Grades didn’t answer at all: 54 runs, three requested efforts, every deterministic score full marks, and a blind rubric judge that couldn’t track the dial. Two days ago we published what Opus 5’s effort dial does on this same battery: identical grades, 2.2x the cost. Same week, same tasks, two dials, and they price entirely different things. K3’s dial moves the thinking. Almost nothing you pay for moves with it.

The setup

Same public arena as the whole series: six graded tasks, byte-identical prompts, hidden graders scoring finished workspaces, three repeats per cell, strictly serial. Pre-registered design pushed before the first graded run. K3 ran in its own vendor’s CLI (Kimi Code 0.27.0) against Moonshot’s platform API, with reasoning_effort requested per model alias: low, high, max, the three values Moonshot’s API documents, max being the shipped default. 54 graded runs cost $5.34.

One instrument matters more than the rest here, and the first smoke run is why. Kimi Code logs the requested thinkingEffort on every API request, so each run carries wire-level proof of what was asked. Smoke run one, labeled low, passed its grade; its wire log said max on every call. The effort setting hadn’t flowed through our config, and nothing else in the run would ever have told us. We fixed it, re-smoked, and only then ran the grid; all 54 published runs prove their requested effort on every request. If an effort setting is supposed to be saving you money, check it at the wire once. The failure mode is silent and it passes all your tests.

Disclosures, this bout’s collection. Home field is reversed from most of the series: K3 runs in its own vendor’s harness, and harness pairing has a sign, here favoring K3. The rubric judge is Opus 4.8, an Anthropic model blind-scoring a rival’s output. Fable 5 co-authors the harness and this article. Moonshot fixes sampling, so the arms differ only in requested effort. Single vendor, single day of serving. And a correction of our own is on the record in the bout’s analysis file: our pre-registration misquoted Moonshot’s launch blog as saying K3 accepts “only” maximum effort. The blog actually said max is the default with low and high to follow, which they did, about two days after launch. The experiment survives the misreading; the gotcha we might have written does not.

What the dial moved

Per 6-task passlowhighmaxlow to max
Thinking (chars)2,2056,93126,66512.1x
Completion tokens6.9k10.5k16.2k2.3x
Execution time358s512s688s1.9x
Tool calls4053561.4x
Cost$0.50$0.59$0.691.4x
Runs passed18/1818/1818/18flat
Kimi K3 effort dial, low to max multipliers: thinking 12.1x, completion tokens 2.3x, execution time 1.9x, tool calls 1.4x, cost 1.4x, grades 1.0x What turning K3's dial from low to max multiplies Thinking volume 12.1x Completion tokens 2.3x Execution time 1.9x Tool calls 1.4x Cost 1.4x Grades 1.0x Dashed line: 1.0x (no change). Bars scaled to multiplier; 54 runs, 3 efforts x 6 tasks x 3 repeats, serial.
Figure: low-to-max multipliers per battery pass. The dial's output is thinking; the bill barely notices.

The thinking is real reasoning, not padding. On the code-review task, the low arm’s first run thought for 906 characters and its report found all four planted defects: the off-by-one pagination, the mutable default argument, the SQL injection, the swallowed exceptions. The max arm’s first run thought for 12,584 characters, a careful file-by-file analysis, and its report found the same four. Both scored 6/6. From the low run’s thinking, in full flight:

1. sync.py:14 — off-by-one: `range(total_count // PAGE_SIZE)` drops the
   last partial page (e.g. 150 users → 1 page fetched, 50 lost). Should
   be ceiling division.
2. sync.py:24 — mutable default argument `seen=[]` persists across
   calls, so users synced in one call are skipped in later calls.
3. sync.py:35 — SQL injection via f-string interpolation of email/id.
4. sync.py:38-39 — swallowing exceptions silently with `pass`

(bouts/2026-07-27-k3-effort-dial/05-review/kimi-k3-low-kimicode/run-1/wire.jsonl.) Fourteen times that thinking bought the same list.

Why the bill refused to follow

The pre-registration predicted cost would order low < high < max on at least five of six tasks. It ordered on four. On the bugfix task the max arm was the cheapest of the three; on synthesis, high cost more than max. That miss is the mechanism showing through: most runs in this bout moved 100k to 300k input tokens (the full spread ran 64k to 407k), most of them cache reads billed at $0.30 per million, against one to five thousand tokens of completion. The bill tracks how many turns the agent takes, because every turn re-reads the context; where a higher-effort arm happened to converge in fewer turns, it was cheaper, effort be damned. The dial governs reasoning volume. The meter counts traffic.

That’s the difference between the two dials this series has now measured. Opus 5’s dial moved completion volume hard enough to move money: its published ladder ran $0.53 at low to $1.19 at xhigh on three tasks, and across the full battery the dial swung cost 2.2x ($1.48 to $3.33) and execution time 3.1x. K3’s dial moves thinking harder than Opus 5’s moves anything, and the bill shrugs: 1.4x, partly non-monotone, 19 cents end to end. Same battery, same grades-refuse-to-move result, opposite economics: K3’s maximum-effort premium is priced in minutes, not dollars. Turning the dial down saves you 330 seconds a pass and pocket change.

(Our K3 launch audition said Claude Code’s effort setting “likely has no effect on K3.” Both statements stand: that was Claude Code’s dial, which most likely dies at the API boundary. This bout turned Moonshot’s own, and proved delivery to the wire.)

The verbosity reputation, measured

The discourse says K3 is a token furnace: Artificial Analysis measured roughly twice the output tokens of same-tier models across their eval suite, and vLLM’s day-zero post says “Kimi K3 thinks a lot before it answers.” We believed it too: the pre-registration bet that K3 at max would emit at least 1.5x Opus 5’s output tokens. It came in at about a third. On the three tasks our Opus 5 ladder ran, K3 at max emitted 6.6k completion tokens where Opus 5 at xhigh emitted 19.0k, computed identically from both bouts’ run metrics; our own prediction inverted, which is what pre-registering is for. That’s a cross-harness comparison (Kimi Code against Claude Code), so treat the ratio as coarse; but it replicates the launch bout’s measurement inside Claude Code itself, where K3 was the smallest talker of three models (3,578 output tokens on the synthesis task against Fable’s 6,145 and Opus 4.8’s 8,001). Suite-level verbosity numbers do not transfer to agentic work. K3 spends its tokens thinking, throws the thinking away, and writes less than either Claude. Budget by workload, not by reputation.

What the judge couldn’t see

The blind judge (Opus 4.8, three samples per run, rubric and deliverable only, never model names or efforts) scored the two rubric tasks across all three arms: low 27, high 24, max 30, out of 36. The middle setting came last, which is not a finding about the middle setting; it is the shape of noise, and we read it as such. The only rubric dimension that moved at all, connecting review findings into compound failure modes, moved as much within the max arm (2, 0, 1 across its three runs) as between arms, and it’s the same dimension four lines of prompt moved more than model choice ever has in this series. 26,665 characters of thinking per pass bought nothing this judge could price.

Which brings up the battery’s ceiling, stated plainly. Two vendors’ effort dials have now run this battery end to end, and 120 effort-arm runs have produced 120 passes. This instrument cannot detect what maximum effort buys, because nothing in it is hard enough to need it. The defensible claim is exactly this: on work the model already handles, requested effort changed reasoning volume 12x and outcomes 0x. Pricing the dial on a problem at the edge of K3’s ability needs a task no model aces. We know; it’s on the list.

The dial to check and the dial to set

K3 ships with the dial at max. On everything this battery can represent, that default buys you a longer transcript and a longer wait: same grades, no depth difference the rubric could detect, 1.9x the execution time, at a cost difference you will not notice on the invoice. Set it low for checkable, tool-heavy agent work; keep max as insurance for problems harder than your eval can see, and know that unlike Opus 5’s version of that insurance (2.2x the bill), K3 charges you mostly in minutes. And before trusting any of it, log the requested effort at the wire once: ours silently reverted to max on the first try, passed everything, and looked identical from the outside.

The serving story moved while we measured: this bout ran on Moonshot’s platform because on weights day it was the only server, and within 24 hours Fireworks, Together, Baseten, and Nebius were live, eight providers by the time this published. Same weights, same dial, eight meters. That bout runs as soon as we hold keys for more than one of them.

Questions this raises

Straight answers.

What does Kimi K3's reasoning_effort setting actually change?
On our six-task graded agent battery, run in Kimi Code with the effort requested per API call and verified in the wire logs: reasoning volume, monotonically and steeply. Thinking went from 2,205 characters per battery pass at low to 26,665 at max, a 12.1x swing. Execution time followed at 1.9x (358 to 688 seconds per pass) and completion tokens at 2.3x. What barely followed was cost ($0.50 to $0.69 per pass, 1.4x), and what never moved was results: 54 of 54 runs passed with full deterministic scores at every effort, and a blind rubric judge could not separate the arms.
Should I run Kimi K3 at low effort for coding agents?
For work resembling this battery, the evidence says yes: the low arm matched max on every grade, with no depth difference the blind rubric judge could detect, while running 1.9x faster and 28% cheaper. The honest boundary: this battery is saturated (two different vendors' models have now aced it at every effort), so it can show that low is safe at this difficulty, not what max buys on harder problems. K3 ships with max as the default, so paying the maximum thinking tax is what happens if you change nothing.
Why doesn't higher reasoning effort cost much more on agentic work?
Because the bill isn't reasoning. Most runs in this bout moved 100k to 300k input tokens (mostly cache reads, billed by Moonshot at $0.30 per million) against 1k to 5k tokens of completion. Cost tracked how many turns the agent took, not how hard it thought: on two of six tasks a higher-effort arm was cheaper than a lower one because it happened to take fewer turns. The dial governs thinking; the meter mostly counts context traffic.

AI Cost Optimization

The writeup has a service behind it.

If this is your situation, the ai cost optimization is where it gets fixed — by the person who wrote this.

Request an audit