AdversarialDebate v0.2.1 adds fail-fast row-count checks after silent join collapse

AdversarialDebate, an open-source framework that runs two LLM reviewers through a structured debate over each other's work, shipped version 0.2.1 on August 28, 2026. The release adds row-count invariant assertions at all five seams of its evaluation pipeline, so the run now halts immediately if rows are silently lost, alongside the project's first false-negative measurements and 55 new unit tests. The developer, writing on dev.to, frames the release as the closing chapter of a two-part debugging story: first a prompt design flaw, then a data-integrity bug in the plumbing that was supposed to measure the fix.

An engine that worked and debates that didn't

The project was built around hard architectural guarantees: isolated reviewer sessions so the second model cannot see the first model's answer, a revelation gate that starts debate only after both reviews are committed, claim and concession tracking, convergence scoring, transcript logging, and a SQLite audit trail. By the author's account, every one of those invariants checked out — and the first real debates were nonetheless inert. In an initial nine-debate run, eight produced zero concessions, the average convergence score was 0.02, and none produced a verdict. Transcripts looked busy while the argument state barely moved; the author's term for the pattern is "debate theater," and he reports it in 89% of that small run.

Debugging instinct pointed at the engine: session isolation, the revelation gate, replayed review text, evidence tracking, convergence math. All were fine. The models were following instructions — and the instructions were the problem.

The zero-cost CARRIED option

The original debate protocol let each side respond to an objection in three ways: concede it, rebut it with counter-evidence, or acknowledge it while keeping the original position, a move the protocol called CARRIED. That third option was under-specified and carried no cost. Nothing required the model to supply evidence for keeping a claim, and nothing forced it to yield when the other side's evidence was stronger — so models took the cheap move and rode out the rounds until the round cap ended the conversation.

Rewriting the prompt to close that loophole changed the system's behavior at scale. After the fix, the author reports zero theater cases in the small run and one in 411 full-corpus debates (0.2%). Average convergence rose to 0.445 in the small run and 0.65 across the corpus; 410 of 411 debates contained at least one concession, with 8,894 concessions, 152 verdicts and 259 disputed outcomes in total.

Then the bug moved down a layer

With the prompt fixed, v0.2.0's failures shifted from debate incentives to whether the evaluation pipeline was faithfully recording what the system actually did. The author lists three issues. Scripts still assumed pull-request-only identifiers after the corpus moved to mixed-domain artifact IDs. Some non-PR artifacts were index pages rather than pinned source documents, ingested as raw HTML and producing garbage reviewer inputs and bad-request failures. Most notably, an LLM-as-judge merge step whose workers correctly judged 2,333 rows had merge logic keyed on the wrong identifier, collapsing the output to 359 rows.

Nothing crashed. The pipeline simply produced results covering roughly 15% of the judged rows and presented them as the finding.

What v0.2.1 changes

Version 0.2.1 adds row-count invariant assertions at all five pipeline seams, which the author says structurally prevents the 2,333-to-359 style of collapse: if rows vanish between stages, the run fails fast instead of producing a confident wrong number. The release also introduces false-negative measurement — a 1.7–3.4% missed-issue rate, the first recall figures the project has reported — plus 55 new unit tests. The package is available on PyPI.

Why it matters

This is a useful case study for anyone building multi-agent or LLM evaluation systems, for two separate reasons. First, a pipeline can pass every mechanical check — sessions isolated, JSON parsed, transcripts written, reports rendered — while the core interaction does nothing of value; when that happens, the bug is likelier in the incentive structure of the protocol than in the architecture, and debugging the engine first will cost you time. Second, silent row loss is the characteristic failure of evaluation plumbing: a join keyed on the wrong identifier does not throw an error, it just makes your metrics lie. Cheap row-count assertions at pipeline seams, together with a willingness to measure recall rather than only headline behavior, are the difference between knowing a system works and shipping something that merely looks like it works.