Coding harnesses may buy tokens more reliably than capability
A new study tested five coding models across Claude Code, mini-SWE-agent and OpenCode on SWE-bench Verified, holding the models constant while changing the harness around them.
The new finding is not simply that harnesses differ. Harness changes generally shifted accuracy by about as much as rerunning the same setup, while task costs varied by as much as three times. The additional spend could fund longer conversations, more tool calls and repeated attempts without reliably solving more tasks.
The study covers only five models, three harnesses and one benchmark. The question for the field is whether harnesses deliver genuine capability gains on other tasks, or mainly make agents work harder. https://spaisee.com/article/coding-harnesses-can-triple-costs-without-making-ai-agents-better?utm_source=nostr&utm_medium=social
SPAISEE
npub1r3…pna4q
2026-10-28 18:09:00 UTC
Author Public Key
npub1r3usm7fjx33mnhrgejw6fg4glna706tkyzlfktgxgec2y2qzdmfqcpna4qSeen on
wss://relay.damus.ioPublished at
2026-10-28 18:09:00 UTCEvent JSON
{
"id": "e304655d226d43e41ce2971ff527978163fd4e29c72e5f46fbdbf2039b480c2a",
"pubkey": "1c790df9323463b9dc68cc9da4a2a8fcfbe7e97620be9b2d064670a228026ed2",
"created_at": 1793210940,
"kind": 1,
"tags": [
[
"r",
"https://spaisee.com/article/coding-harnesses-can-triple-costs-without-making-ai-agents-better?utm_source=nostr\u0026utm_medium=social"
]
],
"content": "Coding harnesses may buy tokens more reliably than capability\n\nA new study tested five coding models across Claude Code, mini-SWE-agent and OpenCode on SWE-bench Verified, holding the models constant while changing the harness around them.\n\nThe new finding is not simply that harnesses differ. Harness changes generally shifted accuracy by about as much as rerunning the same setup, while task costs varied by as much as three times. The additional spend could fund longer conversations, more tool calls and repeated attempts without reliably solving more tasks.\n\nThe study covers only five models, three harnesses and one benchmark. The question for the field is whether harnesses deliver genuine capability gains on other tasks, or mainly make agents work harder. https://spaisee.com/article/coding-harnesses-can-triple-costs-without-making-ai-agents-better?utm_source=nostr\u0026utm_medium=social",
"sig": "b1c3124825f94f52b649cb4e902e8e618dd770a49f38dab5631c7fd6673302274a61b8cb005d04072ddc257de5360c88529d493a01cad2b8d038d7b589c1a654"
}