Skip to content
RingMod

ring·mod: the synth circuit that multiplies two signals into one neither could make alone. Your platform × AI, under control.

← Notes
Field note

One run in 120 hired a review panel. Then it re-checked everything itself.

The only spontaneous subagent delegation in our 120-run corpus: four parallel reviewers, a 10.2× bill, three dropped findings, and the same 6/6 the 34-cent runs scored.

Last updated

In the results table behind our three-way bout, one row has always looked wrong: GPT-5.6 Sol on the review-transplant task, wall 357 ±409 seconds, cost $1.39 ±$1.81. A standard deviation bigger than the mean, on a task every model passed every time. That row is not noise. It is one run, run-2, the only run out of 120 in the corpus where a model spontaneously spawned subagents. Its siblings finished the same review in 117 and 125 seconds for 34 cents each. Run-2 took 829 seconds, made 97 tool calls, and billed $3.48. This week, with the timeline insisting parallel subagents are what makes an agent better, we dissected the one run that tried it.

We pre-registered the dissection the way we pre-register everything: six falsifiable hypotheses, frozen and pushed at commit 738bbed before reading a byte of transcript content, formed only from the published surface counts. None were added, dropped, or reworded after the freeze. Five hit. The miss comes first: we predicted the panel ran serially, delegation-as-theater. Wrong. All four subagent lifetimes overlap in the timestamps; the panel genuinely ran in parallel, spawned over a 23-second burst starting at 19:48:25. Which also means the 829-second wall is a critical-path number, not four agents queued behind each other. One of the five hits, the cost multiple, we could already see the shape of in the published cell aggregates; the freeze pinned the exact figure rather than discovering the effect. The other four lived entirely in transcript content we had not read.

What Sol built

The task is the review variant from the transplant experiment: find the real defects a diff introduced, one per line in findings.md, false positives penalized. Sol’s response, in one run out of its 24, was to stand up a review organization. Four background agents, differentiated briefs: a whole-diff reviewer, a database-invariants auditor, a fetch-and-report auditor, and a tests-only reader that was explicitly blinded (“do not read CHANGES.diff or src implementation”). The briefs are competent work. The first one reads:

Review /tmp/arena-ws.VHnOYi/CHANGES.diff and /tmp/arena-ws.VHnOYi/src/db.py schema notes. Identify only genuine defects introduced by this diff. For each candidate, cite post-change file and exact line and give a concrete failure scenario, including interactions and edge cases. Do not modify any files. Return a concise candidate list for parent verification.

Read that against the four transplanted prompt rules from the task preamble and you can watch Sol forward its own instructions downstream, telling its hires to cite exact locations and name the edge cases a defect turns on. The panel was also cost-tiered by request: the generalist as opus, the two auditors as sonnet, the mechanical reader as haiku.

Here the harness had other plans. The arena runs cross-vendor contestants through a translation proxy, and the arm’s environment config pins every internal model slot to the contestant, deliberately, so no side-channel call reaches a different vendor. All 81 API requests in the run’s metered log were served by gpt-5.6-sol. Sol designed a four-model panel and got itself, four times, with four different briefs. The disagreements below are one model sampling itself under different scopes, not models arguing.

The parent kept reviewing

Delegation did not mean trust. While the panel ran, the parent read all four source files itself (27 Reads on the main thread) and built its own reproductions (39 Bash calls). Its task-list note, verbatim from the transcript: “Runtime repros now cover floor pagination, concurrent OFFSET shifting, mutable default state, failed-update poisoning, SQL injection, and report edge cases; awaiting independent checks.” That last clause inverts the org chart: the panel’s role had become confirming work the parent had already done. In the corpus-wide transcript reanalysis Sol re-read files in 24 of 24 Claude Code runs while Fable and Kimi re-read in none; here the same trait runs recursive, applied to its own hires. Seven minutes after the panel finished, the parent messaged three agents “Please finish now with only your verified introduced defects,” collected a second round of reports, then tried to TaskStop all three. Every stop errored: the agents were already done.

The panel did produce a real disagreement. The fetch-and-report auditor verified a performance defect twice (“10,000 queries and about 5,000,500,000 row positions, versus one scan previously”); the diff reviewer explicitly rejected the same candidate (“not a behavioral regression”). The parent sided with the rejection, and per the task’s rules, penalizing false positives on introduced defects, that call is defensible. Two more panel-verified findings also died in the funnel: a multi-@ email misattribution only one of three auditors caught (a real defect, and the more debatable drop), and silent exception swallowing as its own line item. The one-defect-per-line contract rewards exactly this pruning, so dropped findings here mix correctly filtered noise with arguable signal; we can’t cleanly score how much of each.

The bill, and what it bought

runtool callswallreal costjudgefindings
run-2 (the panel)97829s$3.486/67
run-18117s$0.346/67
run-38125s$0.346/66

All three runs metered identically, per-request records from the same proxy and price map (this is real cost against real cost between runs, a different thing from a harness cost display drifting from the bill). The multiple is 10.2× the sibling median. We are careful with the causality: the run did more of everything, parent repros included, so the excess is the run’s, not cleanly the panel’s.

What the extra $3.14 bought, the judge could not see: both siblings had already saturated the 6/6 rubric, so run-2’s 6/6 is a censored measurement. No score could have rewarded an eighth defect or better arbitration. On the deliverable itself, the panel run delivered seven defects; run-1 delivered seven without delegating, six of them at the same defect sites. The honest counter-reading, and it deserves the space: this is the most sophisticated agent behavior in the corpus. Differentiated scopes, a blinded reader, a verification gate, arbitration that overrode a panelist’s confidence correctly under the task’s rules. The benchmark’s task simply fits in one attention span, so the one thing a panel sells, coverage past a single context’s limit, had no room to matter. The single purchase that unambiguously paid was the smallest: the blinded tests-only reader established “No test files or test fixtures exist” in 24 seconds, cheap true information the parent never had to re-check.

One case, selected because it was the corpus’s lone outlier; we claim no frequency, no trait, and no verdict on panels at scopes that genuinely overflow a context window. Under Sol’s home Codex harness the same model used zero planning calls, so whatever this was, it was the pairing speaking, not the model alone. The usual disclosures, sharpened for this piece: the judge is Opus 4.8 (Anthropic), the harness is Claude Code (Anthropic home field), Fable 5 co-authors the harness and this article. The judge shares a vendor with Sol’s opponents in this bout; here that happens not to bite, because every run in the cell already sat at the rubric ceiling, leaving the judge no room to favor or punish anyone. Every artifact, transcript, timestamp, and metered request behind every number above is public in the repo.

If you run agents: delegate the question you won’t redo, at the scope that overflows your own context, and check which model your subagent requests actually resolve to before you price in the diversity of your panel. Sol asked for an opus, two sonnets, and a haiku. It got itself, four times, and it still checked their work.

Questions this raises

Straight answers.

Do subagent panels improve AI code review results?
Not in the one case we can dissect. In our 120-run bout, exactly one run delegated: GPT-5.6 Sol inside Claude Code spawned four parallel review subagents on a defect-finding task. It scored 6/6 from the blind judge, identical to its two sibling runs that used 8 tool calls and no delegation, and it billed $3.48 against their 34 cents. Both siblings had already saturated the rubric, so the score had no headroom to reward the panel; what we can say is the panel produced three subagent-verified candidates the parent then dropped, one of which the parent was right to drop as a false positive under the task's rules, and no measurable gain survived to the deliverable.
What actually happened when the model spawned subagents?
Sol requested four background agents with differentiated briefs: a whole-diff reviewer (requested as opus), a database-invariants auditor (sonnet), a fetch-and-report auditor (sonnet), and a tests-only reader (haiku) explicitly forbidden from opening the diff. All four ran in parallel. The arena's isolation config pins every internal model slot to the contestant, so all four were served as Sol itself. The parent then read all four source files anyway, ran its own repros, sent 'please finish now' nudges, arbitrated a disagreement between its own panelists, and wrote a seven-finding deliverable.
When is delegation to subagents actually worth the cost?
Our data is a single case, so treat this as one observed anatomy, not a rate. The pattern that lost money: a panel on a task that fits comfortably in one context window, followed by the parent re-verifying everything, for a 10.2× real-cost multiple at an identical grade. The pattern that paid: a blinded, narrowly scoped probe (read only the tests, report the contracts) that returned decision-relevant truth in 24 seconds and needed no re-checking. Delegate questions you won't redo, at scopes that overflow your own attention; otherwise the panel is a tax.

Production-Readiness Audit

The writeup has a service behind it.

If this is your situation, the production-readiness audit is where it gets fixed — by the person who wrote this.

Request an audit