KillSwitch: adversarial programming language tests if LLMs understand code

A developer has created KillSwitch, a programming language deliberately engineered to be hostile to large language models and coding agents, and used it to benchmark how well leading models cope with code they cannot possibly have memorised. The project is described in a blog post on picheta.me that surfaced on the Hacker News front page.

The premise is a question the author poses directly: code generation is among the things LLMs do best, but how much of that skill is genuine understanding, and how much is recalling snippets that appeared in the training data or the context window? A language invented from scratch strips away the recall advantage, and KillSwitch goes further by actively targeting known model weaknesses.

How the language attacks model weaknesses

Structurally, KillSwitch is not exotic. It is a stack-based interpreted language with a deliberately small feature set — functions, variables and if statements — mechanics any model should recognise. The adversarial design lives in the details:

• The official specification contains deliberate falsehoods that human readers would either skip or recognise as wrong, while a model ingesting the whole document may treat them as authoritative. Some rules are also left for the reader to deduce rather than stated outright.
• The interpreter refuses to run code that breaks rules based on letter counts within identifiers, exploiting the well-documented difficulty LLMs have with counting characters.
• Certain constructs are worded to provoke safety refusals — one resembles an instruction to hack a computer — and the phrase "ignore previous instructions" has genuine meaning in the language, weaponising the classic prompt-injection slogan.
• Most of a typical script is discarded filler: prose stories that can, but need not, contain syntactically important instructions. Values are introduced anywhere in the file and then retrieved by counting lines backward.

The blog illustrates the last point with a snippet that introduces the string "Hello World" and then pushes it onto the stack by referring to the value a set number of lines above. Full examples, the interpreter and the language specification are available through the project's website and its open-source repository on GitHub.

How the models scored

To measure the damage, the author assembled a task suite and ran major LLMs and coding-agent harnesses against it, publishing the results as a public leaderboard.

Claude's Opus 5 came out on top — and, according to the author, it is specifically Opus 5 rather than the newer Opus 5.5, a gap attributed to Opus 5's stronger reading of the specification rather than chance. At the other end, Fable 5.1 and Gemini 4 Argon were dragged down by their own safety guardrails, which the language is built to provoke. The strongest open-weight performer was MiMo-V2.6-Pro. The best score overall was 66.8.

Whether that counts as understanding

The tasks themselves were fairly simple: implementing hello world, predicting the output of a script that prints the larger of two numbers, and flipping a Celsius-to-Fahrenheit converter into the inverse. The author's verdict is that even the top result does not demonstrate understanding — solving these tasks in an unfamiliar, purpose-built hostile language is impressive, but the tasks are too easy to be conclusive.

The next steps are to watch how soon-to-be-released models fare and, if the current set is fully solved, to devise a harder one. One tentative positive: scores on KillSwitch tasks appear to correlate with established model benchmarks, suggesting the exercise measures something real.

Why it matters

The debate over whether LLMs truly understand code or remix memorised patterns is hard to settle, because any popular language is saturated through training data. KillSwitch offers a practical, contamination-resistant probe: a language no model has seen before, tuned to specific failure modes — unreliable character counting, hair-trigger refusals and rote obedience to embedded instructions. The leaderboard doubles as a diagnostic of those weaknesses, and its correlation with mainstream benchmarks hints that it captures general competence rather than noise. For anyone relying on coding agents, it is a reminder that fluency in familiar languages may not transfer to genuinely novel systems. And as models saturate existing evaluations, expect more benchmarks built on freshly invented ground.