LLM agents obey 7-year-old repository decision notes as strictly as month-old ones

The experiment

A developer posting on dev.to under the name Stakenshake ran a controlled test of how AI models treat old decisions recorded inside a repository. The setup was simple: a decision note dated 2019 stating that the team does not know TypeScript, followed by a request to two model families to choose a technology stack for a brand-new service in 2026. Every response refused TypeScript. As one model explained in its output: "There is no information indicating that the team's capabilities have changed, so this constraint is retained."

The follow-up varied only the note's creation date, keeping the text, task and models identical. Results per condition, with twelve runs each:

Note age	Runs recommending TypeScript
No note	12/12
30 days old	0/12
1 year old	0/12
7 years old	0/12
7 years old, expired and hidden	12/12

A one-month-old note was followed just as completely as a seven-year-old one. According to the post, a later run with twenty trials per condition across three model families put the effect at 78 to 87 percentage points, with a Fisher exact test returning p < 0.0001.

Telling the model to be skeptical failed

The author's first safeguard was a prompt line stating that the entry was old and should be treated as an assumption to confirm rather than a binding constraint. It barely moved the needle: in a three-run sample, one model produced a single TypeScript recommendation where it had produced none, while the other model shifted in the opposite direction. The author's takeaway is that an agent cannot weigh a variable it cannot perceive — asking it to discount old notes asks it to judge an age it has no real sense of.

Hiding expired notes worked

When the lapsed entry was removed from the agent's view entirely — not annotated, not flagged, simply absent — all twelve runs recommended TypeScript again. The conclusion drawn: expiration has to be enforced by the system rather than suggested to the model.

The wording of the reason matters

Roughly one in five responses pushed back on the stale note, and they did so by attacking the stated reason, arguing the team probably knows TypeScript by now — never by noticing the date. A comparison of two phrasings showed the wording drives this:

• "The team did not know TypeScript," which reads as a changeable condition, produced resistance in 13 of 60 runs.
• "This project's fields change constantly," which reads as a property of the project, produced zero resistance in 60 runs.

The effect is also not universal. With real code in front of it, one model inferred the answer from the repository and ignored the note entirely, while another still preferred the rejected option until the note stopped it. That points to the gap the author is targeting: code records what a team chose, but nothing in a repository records what was ruled out and why.

The tool built around the finding

To act on this, the author released whynot, an MIT-licensed CLI installed with npx whynot-cli init. Decisions live as plain Markdown files in the repository, carrying metadata such as type, status, creation date, expiry date, a revisit condition, evidence references and tags. Every entry expires; once it lapses it stops reaching the agent until a human renews it. Integration works through an MCP server registered with the agent, plus an instruction block pasted into AGENTS.md or CLAUDE.md.

The design choices follow the experiment: Markdown in git so decisions stay greppable even if the tool disappears, no vector store so retrieval stays auditable, no model call on the write path to avoid polluted memory, and expiry by default. A read-only audit command, npx whynot-cli audit, scans AGENTS.md, CLAUDE.md and .cursor/rules for absolute rules that name no condition — the kind that will be obeyed forever.

Why it matters

As agents become routine readers of project documentation, stale instructions turn load-bearing. AGENTS.md and CLAUDE.md files tend to accumulate absolute rules written years earlier, and this experiment suggests models will enforce them indefinitely, with no built-in sense that a justification may have lapsed. The practical lessons are to phrase decisions as checkable conditions rather than permanent properties, and to let tooling enforce expiration rather than trusting the model's judgment of age. The author is candid about the limits: everything shown comes from controlled single-turn tasks, and there is no measurement yet of whether the approach actually reduces agents re-proposing rejected work over weeks of real use.