Four lines of prompt bought the expensive model's judgment

The one visible quality gap in our July 6 model bout was a single sentence. Reviewing a diff with six planted defects, both Claude Opus 4.8 and Claude Fable 5 found all six; only Fable noticed two of them were conspiring. Failures were being “silently swallowed with pass instead of being logged,” it wrote, and combined with a shared list they marked “failed users as done so they are never retried.” Two medium bugs, one severe compound failure mode. That’s the sentence a staff engineer gets paid for, and at the time it was the clearest thing the 2× price premium had bought.

So we tried to steal it.

The transplant

We wrote Fable’s observable habits down as four prompt rules, verbatim from the task variant we published:

• Cite a precise location (file and line) for every claim you make.
• When you report multiple findings, state how they interact: if two defects combine into a worse failure mode than either alone, describe that compound effect explicitly.
• Quantify every comparison with a number computed from the data (a percentage or ratio), never a bare ranking.
• When a correct answer depends on an edge case, name the edge case and state how you handled it.

About a hundred tokens. Then three arms, five runs each, on the review task and a constrained-reporting task: Opus as-is, Opus with the preamble, and Fable as-is as the bar to clear. Deterministic graders scored correctness as usual. Quality went to a judge model that saw only a rubric and the anonymous text, never a model name, and scored each run three times; we report medians. The rubric and the arms were committed before any transplant run, along with five predictions about how it would go.

What the judge saw

Five of six rubric dimensions came back at ceiling for every arm. Vanilla Opus already cites file and line, already explains failure mechanisms, already writes grounded recommendations. The whole experiment came down to the dimension that started this: does the review connect defects into compound failure modes?

Per-run scores on that dimension, 0 to 2:

Arm	Five runs, sorted
Opus 4.8, vanilla	0, 0, 1, 2, 2
Opus 4.8, transplant	0, 2, 2, 2, 2
Fable 5, vanilla	0, 2, 2, 2, 2

The transplant distribution matches Fable’s exactly, down to each producing one whiffed run. Synthesis turns out to be something vanilla Opus does sometimes, Fable does usually, and prompted Opus does exactly as often as Fable. On these two tasks the judgment gap was a prompting gap.

Note what the table also says: none of the three arms does this reliably. Five runs is a small sample and a 0–2 rubric is a coarse instrument. We’d call this a strong result on one behavior, measured once, in one domain, with a same-family judge, and we’ve published every judge sample alongside the analysis so you can discount it yourself.

The prediction we got wrong

We pre-registered five hypotheses and one missed. The prediction: since the preamble adds only about a hundred input tokens, transplant cost would stay within 15% of vanilla. Measured: 28% over ($0.74 against $0.58 for the two-task pass), with output tokens up 41% on review and 51% on reporting. Asked to defend edges and quantify comparisons, the model writes more. There is no way to have the behavior without the tokens.

Which reframes the premium we started with. Fable produced its compound- failure sentence while costing $0.99 on the same two tasks. Some of that gap against vanilla Opus’s $0.58 was never the rate card; it’s the token bill of the habits themselves, and it follows the habits to whichever model performs them. Prompted Opus at $0.74 is the same judgment at 75% of the price, and that ratio, unlike a latency number, has held up well under repeats.

Where this leaves the buying question

Before paying a 2× rate for a model’s judgment, write down what the judgment looks like in the transcripts. If it decomposes into rules a preamble can state, try the preamble first and budget for the extra output it triggers. What survives that transplant is the part of the premium worth arguing about, and on harder, longer tasks than these there may be plenty. Our tasks stop discriminating early; that’s the next bout.

Turning “the expensive model feels smarter” into a rubric, three arms, and a blind judge took an afternoon; the ten new agent runs cost $3.70, the judge calls a dollar or two more (we didn’t meter them, a gap the analysis flags). The production-readiness audit builds this kind of harness around your workload, where the answer actually matters.