Drew Breunig's state of context engineering talk maps four LLM context failure modes

A state-of-the-discipline talk

Drew Breunig used a short talk at the AGNTCon conference this week to deliver what he describes as a state-of-the-discipline assessment of context engineering. The talk, titled "Context Fails & Fixes," traces where the practice came from, the failure modes it must now manage, and the posture builders should take; Breunig published the slides and a transcript on his blog, and the post has circulated widely since reaching the Hacker News front page.

The organizing idea, stated in the talk's opening line, is that the job now is judging when to rely on deterministic code and when to lean on the probabilistic model.

A discipline born from the long-context bet

Breunig — whose O'Reilly book, The Context Engineering Handbook, is forthcoming — traces the discipline to late 2024, the peak of the "RAG is dead" argument. The hope then was that context windows could grow large enough to hold everything, removing the need for selective retrieval or agentic scaffolding. Model makers jumped from roughly 128,000-token windows to million-token windows, starting with Gemini 1.5; Google's marketing for Gemini 1.5 Pro centered on the million-token figure and teased a 10-million-token model in the lab. Meta's Llama 4 Scout shipped 10 million tokens first, but Breunig notes inference was difficult, the model never caught on, and no further Llama arrived. The million-token baseline stuck — and, in his telling, more context simply created more room for problems.

Four failure modes

Breunig groups long-context problems into four. Context poisoning is incorrect material — a hallucination or a bad fact — entering the context, where the model cannot get past it. Context distraction is the window growing so long that the model privileges its contents over its base knowledge. Context confusion is unrelated facts seeping in and steering the model in ways that are poorly understood; his example is a model forming assumptions about who the user is, which then shape answers. Context clash is tool calls, skills and other harness components interfering with one another in unexpected ways.

He also flags assistant memory as a familiar case of confusion and clash: he keeps memory turned off in ChatGPT and Claude because the models take notes across his work and personal conversations, blend them, and compound the confusion.

Tea and the guard

His favorite illustration of poisoning comes from gaming. Pokémon Red (1996, Game Boy) and Pokémon FireRed (2004, Game Boy Advance) are near-identical, except that the bribe needed to get past a security guard changed from lemonade in the original to iced tea in the remake. According to the Gemini 2.5 technical report cited in the talk, researchers used the game as an agent benchmark, and the model — playing the older game — repeatedly convinced itself it needed tea, because the FireRed tea task sat in its weights. Once tea landed on its task list, the model spent hours trying to obtain it and could not let go.

Instructions lose to examples

For distraction, Breunig points to a paper by Carolina Camassa and Derek Shiller titled "Do as I Say, Not as I Do." The setup: instruct a model to answer every question by translating its answer into Russian, then fill the chat history with question-and-answer examples in French. No new instructions — just counter-examples sitting in the history. Every model tested eventually drifted off its instructions: Gemma 3 12B complied less than half the time after only a few contradictory examples, while Opus 4.6 held out longer but degraded as well. The practical implication is that a long working session — say, iterating on front-end CSS — quietly seeds exactly these counter-examples into the context.

Breunig's own habit is to watch the token counter and reset around the 200,000–300,000 mark before starting a big task. When he polled the AGNTCon audience on whether they let coding contexts run all the way to a million tokens, about half the room raised hands — more, he admitted, than he expected.

Why it matters

The talk works as a compact rebuttal to the idea that window size solves the context problem. Even frontier models degrade as the window fills with contradictory examples, and even a correct association baked into a model's weights can derail an agent once it enters the working context. That makes context engineering a durable discipline rather than a stopgap: choosing what enters the window, when to clear it, and which parts of a pipeline should be deterministic code versus model judgement is now core agent-building work — and on Breunig's argument, it will stay that way however capable models become.