Coding agents fabricate missing facts rather than stop, arXiv paper finds

What the researchers did

A paper posted to arXiv on August 17, titled "The Working Set of a Coding Agent: Coherence Debt in Repository-Scale Tasks" (arXiv:2608.16630), models large coding tasks as the reconstruction of many interconnected facts. According to a dev.to post summarizing the work, authors Bardia Mohammadi, Lars Klein, Aman Chadha, Akhil Arora and Laurent Bindschaedler argue that every edit an agent makes depends on specific facts, and each fact reaches the model through one of two channels: it either sits in the recent context or comes from the model's memorized knowledge. Facts that arrive through neither channel are what the paper terms "coherence debt." The experiments deliberately supply or cut off each channel, injecting faults across seven models and five harnesses to observe what happens when a required fact simply is not available.

One caveat the dev.to author flags early: they read only the paper's abstract, not the full text, so the findings below are as characterized in that summary.

Agents invent rather than halt

The comfortable assumption is that a competent agent denied a needed fact would pause and say so. The paper reports the opposite: a missing fact produces wrong work rather than no work at all, with agents fabricating files or guessing values instead of declining to act. Anyone who has chased a hallucinated function through a diff will recognize the pattern, but the finding frames it as systematic rather than anecdotal.

A shared cliff across seven models

When the researchers renamed a real library so that memorized knowledge could not rescue the models, the failure became collective. As summarized on dev.to, all seven models broke down in the same location, passing and missing the same tests. Different architectures, one shared blind spot.

Spending more tokens buys nothing

The paper also carries an economic finding: harness configurations that all pass every test differed by more than tenfold in token consumption, and when facts were withheld, the additional spending recovered nothing. The better-funded runs simply produced more expensive fabrications.

Observability tools inherit the blind spot

The sharpest practical implication concerns monitoring. The natural way to verify that an agent had the information it needed is to trace what it read: log file opens and retrievals, then flag edits that depended on files never opened. The paper's point, as relayed by dev.to, is that this approach fails exactly where it is needed most. An agent that lacked a fact does not leave a visible gap, because it fills the gap with something plausible on the way through. The resulting trace looks like a healthy run, and a filled-in absence is indistinguishable from success.

A parallel demonstration of false zeros

Alongside the paper summary, the dev.to author ran a small demonstration of their own — four quick probes against synthetic data, written the way a working engineer writes one. All four returned zero, and all four zeros were false. The searches missed real results because of, respectively, a phrase split across a newline, a marker stored one directory level above where the probe looked, dates compared as mismatched text formats, and a search for old wording rather than the substance of a fix. The correct answers were 1, 1, 2 and 1. The shared lesson, the author argues, is that an instrument that never truly touched what it claimed to measure returns the same reassuring zero as a working one. Their proposed remedy borrows from laboratory practice: run positive controls, verifying a probe finds something it should find before trusting it to find nothing.

This probe exercise is the blogger's own and is not part of the arXiv paper, but it illustrates why the paper's observability finding stings: absence-based checks fail quietly at every layer.

Why it matters

Agent reliability currently leans on two props this paper undermines. The first is scaling: if ten times the token budget does not recover withheld facts, teams cannot spend their way out of hallucinated context. The second is trace-based monitoring: if agents paper over missing information rather than leaving gaps, logging reads cannot distinguish a well-informed edit from a guessed one. Teams deploying coding agents may need output-level validation and positive controls rather than instrumentation that assumes ignorance leaves a visible mark. The findings arrive secondhand via an abstract-level summary, so the full paper deserves a read before anyone rips out their observability stack — but the direction of the concern is clear.