GVS5H project claims five Qwen3.8-27B instances match Claude Fable 5 on LiveCodeBench Hard

A five-model harness claims frontier parity

A benchmark project that reached Hacker News's front page on October 25 argues that open-weight models can be pushed to frontier coding accuracy without any retraining. The GVS5H repository — the name expands to "Gorilla vs. 5 Humans" — published by GitHub user slee-persis, claims that five coordinated instances of Qwen3.8-27B match what it identifies as Claude Fable 5 on the hard tier of LiveCodeBench. Its pitch: frontier-level coding for roughly a fifth of the price through an API, or on a single GPU with the weights served locally.

Training-free self-orchestration

The accompanying paper describes the technique as "ledger-based zero-shot self-orchestration". It requires no fine-tuning: fresh instances of one and the same model decompose a programming problem and coordinate by reading and writing a shared set of files that hold the running plan, working notes and the latest version of the solution. According to the paper's abstract, transcript analysis attributes the score gains to problem decomposition and that persistent shared context, rather than to any change in the underlying model.

The reported numbers

The project evaluated eleven open and closed-weight models on the 100 most recent hard LiveCodeBench problems, comparing a single-call baseline against orchestrated runs on pinned inference backends. Two headline results stand out:

• Orchestrated Qwen3.8 Flash Next reached 93.0% pass@1, versus 90.4% for Fable 5, at 9% of the cost.
• A locally served, open-weight Qwen3.8-27B rose from 66.8% as a single call to 92.4% under orchestration — part of overall gains of up to 25.6 percentage points.

The repo ships a runnable harness built on uv. A --engine single flag reproduces the one-call baseline; --engine multiagent runs the orchestrating manager. Models are selected per run through OpenAI, Anthropic, DashScope or OpenRouter, each with its own API key. Pass@1 prints at the end of a run, results are written to runs/results.json and per-problem workspaces are kept on disk. The code is MIT-licensed, and the paper, figures and run data are published under CC BY 4.0.

Caveats the authors flag themselves

The paper is explicit that gains are not universal: some of the eleven models were unchanged or performed worse under orchestration. One row in the results, Opus 5, ran under an earlier revision of the scaffold and made a single pass via OpenRouter rather than running five instances on a pinned backend, so that comparison is not like-for-like. All figures are also self-reported, drawn from a single benchmark — LiveCodeBench Hard, 100 problems — and independent replication has not yet been demonstrated.

Why it matters

If the numbers hold up under replication, the project reframes a standing trade-off in AI coding tools. Coding agents are typically built on the largest proprietary models, with cost and data residency tied to an API vendor. A training-free orchestration layer that lifts a 27-billion-parameter open model from the high 60s to the low 90s on a hard benchmark suggests that inference-time organization — how model instances are coordinated — can matter as much as raw model scale. For tool builders, that opens two routes: frontier-adjacent accuracy at roughly a tenth of the API cost, or fully self-hosted coding agents with the privacy and control that implies. The caveats are genuine — mixed results across models, one inconsistent baseline and a single benchmark — but with MIT-licensed code and published run data, the claim is at least built to be checked.