Auditioning a replacement for the model that's leaving

Fable 5 is leaving Anthropic’s subscription plan, so I need a successor picked before the change lands, and the leading candidate shipped this week. Moonshot’s Kimi K3 is a 2.8-trillion-parameter open-weight model priced at $3 per million input tokens and $15 per million output, the same list price Anthropic charges for Sonnet 5. It also speaks Anthropic’s Messages API natively, which means it runs inside Claude Code, inside our existing arena, with a small environment file and no other changes. The audition was 72 graded runs: the same six tasks the Opus and Fable bouts used plus both transplant prompt variants, three repeats per cell, strictly serialized. The challenger went 24 for 24 at $2.14 per full pass. So did both incumbents.

The decision got harder, not easier.

The setup

The harness is the same public arena the whole series has run on: byte-identical prompts per task, hidden graders the agent can’t reach by construction, pinned CLI version (2.1.212), pinned effort, one run at a time so nothing shares rate-limit headroom. The only new machinery is a per-model environment file that points Claude Code at Moonshot’s Anthropic-compatible endpoint, plus a check that greps every published transcript and workspace for the API token before anything leaves the machine, because the agent under test can read its own environment and I would rather learn that from a grep than from a stranger.

Disclosures, since this bout collects them. The harness runs inside Claude Code, tooling built by and for one side of the comparison, so home-field advantage sits entirely with the incumbents. The effort setting we pin (xhigh) is a Claude Code concept; K3 thinks by default and most likely ignores it. Moonshot bills cache reads at $0.30 per million tokens against different Anthropic caching mechanics, so treat cross-vendor dollar figures as close, not exact. The rubric judge is Opus 4.8, which is also a contestant. And the model leaving the subscription plan helped write this article. Every transcript, grade, diff, and dead run is in the public bout directory if you’d rather check than trust.

The scoreboard

Model	Runs passed	Cost per full pass	Wall clock per pass
Kimi K3	24/24	$2.14	1031s
Opus 4.8	24/24	$2.27	621s
Fable 5	24/24	$4.56	609s

Every deterministic score came back full marks, all three models, all eight tasks. Whole bout: about $29 in API spend, of which $2.11 went to runs an overloaded API killed. More on that below.

K3’s profile is cheap, slow, and quiet. Cheapest per pass by 13 cents over Opus and less than half of Fable. Slowest by a wide margin: 17 minutes for the course against roughly 10 for either Claude, a 1.7× gap that held on every task. And the smallest talker of the three: on the synthesis task K3 emitted 3,578 output tokens where Fable spent 6,145 and Opus 8,001. The extra minutes are not verbosity; the model says less than either incumbent and takes longer to say it. The time lives in Moonshot’s serving, not in the transcript.

Where behavior overlapped, it overlapped strikingly. On the refactor task the three models took 14, 14, and 15 turns for the same grade, and K3’s three runs each finished in 80 seconds, to the second, at 24 cents, to the cent.

Then the launch-day traffic arrived

Our bout started at 04:59 UTC, which is 12:59 in Beijing, lunchtime on K3’s release day. The first three tasks ran clean: nine Kimi runs, nine passes, zero retries. A single retry appeared in the fourth task. By the sixth, every Kimi run was retrying against the same response: Request rejected (429) - The engine is currently overloaded, please try again later. Five runs burned all ten of the CLI’s retries and died after thirteen and a half minutes apiece, one assistant turn, zero tool calls, empty workspace. The graders scored the empty workspaces the only way graders can (FAIL: REPORT.md missing), which is exactly why those grades can’t be read as model failures. The model never got the chance to be wrong. The two Claude models, running interleaved on the same machine over the same hours, logged zero retries in 48 runs.

We archived the eleven contaminated runs in the bout directory under _launch-day-429/, reran those cells at 16:51 UTC, just after midnight Beijing time, and got eleven passes with zero retries. The scoreboard above is built from the clean cells; the corpses are published next to it.

Both facts matter to the buying decision. The model qualifies: nothing in 24 clean runs distinguishes its correctness from the frontier. The sole hosted provider, on day one, did not: a coding agent that dies for thirteen minutes at a time during Beijing business hours is not a daily driver in a US-evening workflow, whatever the weights can do. The escape hatch is the license. The weights are open, other providers will serve them, and an overloaded launch-day API is a queueing problem, not a capability ceiling. We also benchmarked a serving fleet on its worst possible day, hours after release; rerun the reliability half of this in a month and it may be unrecognizable.

Everyone passed everything, which is its own finding

Eight days ago this battery still discriminated: Haiku 4.5 invented revenue totals twice in five runs and the cheap rung broke. Today a day-one open-weight release ties the two models this series was built around, full marks everywhere. Two readings fit that result, and I can’t fully separate them with this data.

Reading one: the tasks sit below all three ceilings, so the battery now measures a floor that a $2.14 model clears as cleanly as a $4.56 one. We half-admitted this in the ladder bout when three of four models cleared everything, and the spread has only narrowed since. Reading two is more interesting for anyone choosing a model by benchmark: Claude Code itself, the tool scaffolding, the prompts, the agentic loop, does enough of the work that the model behind it matters less than the price sheet implies. The 14, 14, 15 turn counts on the refactor task read like three drivers following the same GPS. If the harness is doing the delegation well, that convergence is precisely what good tooling should produce, and the cheapest adequate model becomes the rational default.

The rubric judge was the one instrument left that might separate the three, since it prices qualities the pass/fail graders can’t see: citation precision, quantification, synthesis across findings. It came back nearly as flat as the graders. Out of 72 rubric points per model (four tasks, three dimensions each, three runs, scored 0–2 by an Opus judge shown only rubric and deliverable, never model names), Fable took 70, Opus 68, Kimi 67. Every dimension but one was full marks for all three models. The one that moved, interaction synthesis on the code-review task (does the write-up connect separate defects into compound failure modes), is the same dimension our transplant experiment showed responds to four lines of prompt. It responded again here, on all three models: under the baseline prompt the run medians were Fable 2/0/2, Opus 0/0/2, Kimi 1/0/1; under the transplant prompt Opus and Fable went to straight 2s and Kimi to 1/2/2. The prompt moved that dimension more than the choice of model did, which replicates the transplant finding across a vendor boundary and adds a point to the harness-does-the-work reading. A three-point spread on medians-of-three is noise, and I’m not ranking on it.

Either way the practical conclusion for my subscription question survives: on this workload, correctness is no longer the axis of decision, and depth quality barely is. Price, latency, and serving reliability are.

The verdict

My test was never “is K3 the best model.” It was “when Fable 5 leaves the plan, what runs my daily agent work.” On that question, three answers with different shapes.

K3 passes the capability bar today, costs the least per pass of anything we’ve measured above the Haiku rung, and hands you a seven-minute tax on every full pass plus, this week at least, an API that folds under its home market’s afternoon load. Opus 4.8 remains what the last three bouts said it was: same grades as Fable at half the cost, ten-minute passes, and it never once made me think about its infrastructure. Fable 5’s premium bought nothing these graders can price, and the graders are the part of this article you should trust most.

The honest schedule: run your own workload against K3 in a month, when the launch surge has settled and other providers are serving the weights, and let the meter and the stopwatch vote; that walk is the first thing a production-readiness audit runs after standing the harness up. If your bake-off has one lesson from our July, it’s that single runs measure weather; the 429s were weather too. Weights this capable at this price will get reliable serving. The only question is whether that happens before the subscription change does.