One-dollar experiment finds repeating a prompt instruction up to four times lifts LLM compliance

A roughly one-dollar weekend experiment has put hard numbers on a familiar piece of prompt-engineering folklore: repeating an instruction in a system prompt genuinely improves how often a model follows it, but the benefit stops at around four repetitions. The results come from a khola.blog post titled "Say It Four Times (In Your System Prompt)", published on 19 August 2026 and picked up on the Hacker News front page.

How the test worked

The author picked a single rule a model can either obey or ignore — write Python strings with single quotes, never double quotes — and requested six ordinary functions, such as merging intervals or flattening a dictionary. The only variable across runs was how many times the rule appeared in the system prompt: zero, one, two, four, eight or sixteen.

Each combination ran thirty times, 1,080 generations in total, all on Gemini 2.5 Flash via Vertex with thinking mode disabled. Scoring required no judgment: Python's own tokenizer counted strings that opened with a double quote.

Rule selection was its own lesson. Three earlier candidates — no comments, no docstrings, no type hints — were already followed about 99% of the time on first mention, leaving no room for repetition to show an effect. Quote style was chosen precisely because the model resists it.

The numbers

The control anchors everything: with no instruction at all, the model used double quotes in all 171 usable zero-repetition generations. Stating the rule once lifted compliance to 74%. Four repetitions brought it to 97%. Past that the curve flattened, with eight and sixteen landing in the same range as four — though the author notes a small decline could escape notice at this sample size.

Averages also hid wide per-task gaps. Two of the six tasks hit full compliance on the first mention and stayed there, while the interval-merging task managed only 20% with one mention and needed four to reach 97%. In the author's reading, repetition does not make the model broadly more obedient; it rescues the specific spots where an ingrained habit fights the instruction.

An accidental match with prior research

The experiment was prompted by Han-yu Wang's arXiv paper "When More Becomes Less: Position-Dependent Repetition Effects in Language Models", which reports that repeated copies placed next to each other climb and then plateau, while copies spread away from where the model reads out rise to an early peak and then fall.

The author guessed the experiment would show that hump, and was wrong. Because every copy sat in a single block, the setup matched the adjacent case — the one predicted to climb and flatten. The result therefore lined up with the paper's prediction from an unexpected direction, using a natural-language rule on a coding model rather than token-level probes.

Noise and stubborn leftovers

Between half and two-thirds of the task-and-repetition combinations returned mixed results across thirty identical runs: same prompt, same model, same settings, different answers. The implication the author draws is that judging a prompt change with one run before and one after is close to meaningless — three runs is a floor, ten is better.

The surviving violations had a shape. Once the rule appeared at least once, ordinary double-quoted strings disappeared entirely; what remained were triple-quoted docstrings. The model appears to treat docstrings as documentation rather than strings, so a rule about quoting never reaches them.

Practical guidance

For teams maintaining prompt libraries, the post suggests capping literal repetition at about four and spending the leftover context on examples. For individuals, repeating a stubborn rule up to four times is described as the cheapest available fix — but if four does not work, further copies bought nothing, so the approach should change. When a constraint is obeyed everywhere except one location, look for something the model has categorized differently, as with docstrings.

Caveats and a measurement trap

The author is candid about scope: one person, one model with thinking disabled, one syntactic rule repeated literally in one block, and six standalone functions rather than a real repository or agent loop. Spacing copies throughout the prompt — the configuration the paper predicts will eventually hurt — remains untested.

The first real run also nearly sank the experiment. Gemini's thinking tokens counted against the output limit but were reported separately, so code was silently cut off mid-generation, and truncation hit some conditions harder than others. Before the author caught it, the headline compliance figure read 39%; after the fix, 88%. An evaluation setup that does not record why generation stopped can hand back a confident wrong answer.

Why it matters

Prompt-engineering advice circulates widely but almost never with measurements attached. This experiment attaches numbers to one of the most common practices and finds the folklore partly right, with a ceiling. The methodology lesson may matter just as much: at current noise levels, single-run prompt evaluation is unreliable, and harnesses that ignore why generation ended can skew results invisibly. The author has published the guess recorded before running, along with the code and data, for others to verify.