Fable 5 vs GPT-5.6 Sol vs Kimi K3 across 120 graded agent runs

My daily agent work runs on Claude Fable 5, which is leaving Anthropic’s subscription plan, and the two credible replacements are Moonshot’s Kimi K3 and OpenAI’s GPT-5.6 Sol. We put all three through the same eight graded agent tasks: bug fixing, function synthesis, refactoring, environment repair, code review, and constrained reporting, plus two prompt variants, three repeats per cell, and each cross-vendor candidate ran in two harnesses: 120 runs in total. Every run passed. Every deterministic score came back full marks. What separates these models now is not whether they can do everyday agent work; it’s a 3.7× spread on the bill, a 2.9× spread on the clock, and what a blind judge thought of their prose. The most useful thing in the data isn’t on the scoreboard at all: the tool logs show identical grades concealing entirely different ways of working, and the second round of testing showed the working style belongs to the model-harness pairing, not to the model alone.

Model	Runs passed	Cost per full pass	Wall per pass	Tool calls, refactor	Judge (of 72)
GPT-5.6 Sol (Codex CLI)	24/24	$1.25	305s	5	58
Kimi K3 (Kimi Code CLI)	24/24	$1.48	1586s	18	63
Kimi K3 (Claude Code)	24/24	$2.04	891s	13	66
Fable 5 (Claude Code)	24/24	$4.62	690s	12	70
GPT-5.6 Sol (Claude Code, proxied)	24/24	$4.46	1085s	32	60

How each model was tested

• Fable 5 ran in Claude Code (CLI 2.1.212), its own vendor’s tooling, effort pinned at xhigh. Home advantage is real and disclosed; every other row should be read with it in mind.
• Kimi K3 was tested twice. In Claude Code it ran against Moonshot’s own Anthropic-compatible endpoint: same CLI and prompts as Fable, byte for byte. Then 24 more cells ran under Moonshot’s own Kimi Code CLI (0.27.0, metered platform key, predictions committed beforehand). Its two configurations split the honors: cheaper under its own CLI, faster under Claude Code. Both rows stand.
• GPT-5.6 Sol was tested twice as well, and it’s the reason both cross-vendor candidates got second looks. Inside Claude Code (through a local LiteLLM proxy, since OpenAI serves no Anthropic-compatible endpoint) it was slow, expensive, and erratic. Rather than publish that as the model’s character, we reran all 24 cells under OpenAI’s own Codex CLI to give it its best chance, with predictions committed beforehand. Both configurations are published; the comparison uses its Codex numbers.

Everything else is held constant: byte-identical prompts, hidden graders the agent can’t reach, serialized runs, zero API retries anywhere. The rubric judge is Opus 4.8, sampled three times per run and blind to model names; it is an Anthropic model scoring competitors’ prose, and Fable co-authored this harness and this analysis. Every transcript, grade, diff, and judge sample is in the public arena repo.

Price and speed

Sol under Codex is the cheapest and fastest configuration this arena has graded: $1.25 per full pass, 305 seconds, on 82% cached input at OpenAI’s $0.50-per-million cached rate. Kimi K3 splits itself: $2.04 and 891 seconds in Claude Code, $1.48 and 1,586 seconds under Kimi Code, whose more deliberate style bills fewer uncached tokens (93% cache hits) and takes nearly twice as long. Either way its serving stayed boring: zero retries across all 48 Kimi runs, confirming the 429 storms we published earlier were launch congestion, not a property of the service. Fable 5 at $4.62 is 3.7× Sol’s price for the same grades, and its one remaining crown, the 690-second wall clock, fell to Sol’s Codex configuration by a factor of 2.3.

Effort shape tells the same story more precisely. On the refactor task, Fable makes 12 tool calls, Kimi 13 in Claude Code and 18 under its own CLI, and Sol 5 under Codex. Per pass-through, Sol reads 1.9 million input tokens, Fable 1.28 million, Kimi 1.38 million in Claude Code and 1.79 million at home, and every configuration pays cache rates on the large majority of them. None of this shows up in a pass/fail column, and all of it shows up on the invoice.

Working styles, from the tool logs

The transcripts show three distinct ways of doing the same work.

Fable 5 edits surgically and repeats itself almost exactly. Its tool mix across the bout is balanced (81 Bash, 61 Read, 31 Edit, 26 Write), and its three refactor runs produced byte-identical diffstats: 19 insertions, 15 deletions, three times in a row. It also has the steadiest clock: median wall-time variation between identical runs is 16% for Fable against 25% for Kimi and 27% for proxied Sol. What the premium buys, on this evidence, is predictability.

Kimi K3’s paragraph is the one the second round of testing rewrote, so here is the correction in the open. In Claude Code, Kimi rewrites where the others edit: 33 whole-file Writes against 10 in-place Edits across its 24 runs, and we first published that as the model’s style. Under its own CLI the ratio inverted: 41 Edits against 30 Writes, plus 82 Reads, 18 tool calls on the refactor against 13, and nearly double the wall clock. The rewrite habit belonged to the pairing, not the model. What survived both harnesses is less colorful: full marks everywhere, mid-pack depth scores, and serving that never retried once. The practical advice survives too, redirected: if your review process reads diffs, budget for the pairing’s style, and measure it in your harness, because ours mislabeled it from one.

Sol under Codex batches. It averages 5.5 tool invocations per run (96 command executions and 36 file changes across all 24), reads the least, and finishes whole tasks in a single prompt-to-completion cycle. The economy of motion is what makes it cheap and fast; it also means there’s little intermediate state to inspect while a run is in flight, which matters if your workflow supervises agents mid-task rather than reviewing finished work.

Prose depth is where they actually differ

The judge separated the three: Fable 70 of 72, Kimi 66, Sol 58. The rationales it wrote are more useful than the totals.

Sol is precise and shallow at once. It took full marks on specificity and location precision in every configuration: its recommendations name a product, a figure, and an action, and its review findings cite file and line. What it doesn’t do is compute. On the report task the judge scored it 1 of 2 on quantification in every run, in both harnesses, with rationales like “raw totals attached to rankings rather than computed comparisons,” and 0 on insight for prose that “only restates the top/bottom rankings already implied by the tables.” Fable’s 2s on the same task came from derived claims: “West trailed at $1,414.82, roughly 35% below North,” and a median-versus-mean comparison the tables don’t state. Same correct figures underneath; one model argues with them, the other reports them. Kimi sits between, with a mild version of the same profile: full specificity, occasional dropped points on quantification and insight.

Asked to review code with the plain prompt, Sol also scored zero on connecting defects into compound failure modes.

The repair is cheap and known. The four-line transplant prompt from earlier in this series took Sol’s interaction-synthesis medians from 0/0/0 to straight 2s, in both harnesses, and has now done the same for four models across three vendors. If depth is what you’re buying, audit the prompt before the model.

The two-harness note, because it generalizes in both directions

Sol’s two rows are the same model. Between them: 3.6× on cost, 3.6× on wall clock, 32 tool calls against 5 on the same refactor. Kimi’s two rows are the same model too, and they move the other way: its own CLI made it cheaper and slower, not leaner and faster, and inverted its edit style. One experiment in each direction retires the simple story. A harness doesn’t uniformly help or hurt its vendor’s own model; it induces a working style, and the effect has a sign per pairing. A cross-vendor agent benchmark measures those pairings, not the models, and the check costs a few dollars if your graders score the finished work rather than the transcript. Both reruns were pre-registered before their first graded run; the Kimi design deliberately put this article’s original working-styles claim at risk. It lost, which is what the method is for.

Verdict

For the replacement question this series exists to answer: Sol plus Codex wins on price and speed by margins that aren’t close, carries one real and prompt-fixable weakness in prose depth, and costs you Claude Code as the daily driver, which is not a small thing when the harness half of the pairing is what this data says it is. Kimi K3 is the drop-in answer: same CLI, same workflow, half of Fable’s price, with serving that has been clean since launch week; its own CLI shaves the bill to $1.48 if you’ll trade the wall clock for it. Fable 5 keeps the best depth scores in the arena and, for as long as it stays on the plan, the luxury of not having to choose.

Run your own workload before the change lands, and read the tool logs, not just the pass rate: the working style is the part of the model you’ll live with daily, and it’s invisible in every leaderboard number. That walk is the first thing a production-readiness audit runs after standing the harness up. Single runs measure weather, three repeats is the floor, and if your candidate model looks bad in your harness, spend the extra $4 before you blame the model.