LLMs tested on TIS-100 claim 133 leaderboard frontier points and 3 cycle records

A puzzle game as a programmatic benchmark

TIS-100, Zachtronics' assembly-language puzzle game, gives players a grid of tiny processors. Each node has two registers and room for fifteen instructions, and neighbouring nodes must wait on one another to exchange values. Working solutions are scored on cycles, nodes and instructions, and pushing one score down usually pushes another up, so each puzzle has a frontier of trade-offs rather than a single best answer. Players have spent roughly a decade refining programs on a public community leaderboard.

According to the TIS-100 Bench results page (dated 3 November 2026, and surfaced on the Hacker News front page), sixteen LLMs — spanning leading frontier and open-weight models — were given the game and its manual, but not the leaderboard, and told to keep improving. Each model attempted all 47 base-game puzzles plus eight community tournament puzzles in one scored attempt (pass@1), with 10 to 45 minutes per puzzle depending on difficulty.

The harness mirrors what a human player can inspect, via text tools instead of a mouse: puzzle specs, visible test inputs, registers and game feedback. Models edit programs and run them in the real game; failed runs return mismatches, register states and blocked ports, so the next edit can respond to evidence. Passing programs are saved and recoverable, so a risky edit cannot lose earlier progress, and reminders nudge a model to keep testing if it tries to quit early. Community code and exact records were withheld; models saw only the game's coarse score histograms.

Headline numbers

Across 55 puzzles (19 easy, 17 medium, 19 hard), the models produced 133 new frontier points — score combinations no community solution had reached — spread over 18 puzzles, plus three outright cycle records. Eighteen of those points beat a community record outright; the other 115 were new trade-offs between the three scores. A point only counted after a fresh-game replay and 100,000 passing tests on an independent simulator, TIS-100-CXX.

Opus 5.5 and GPT-6 Astra lead, and the data does not separate them. Both solved all 55 puzzles and both found new frontier points on 11 of them. Opus 5.5 found more points, 72 to 32; GPT-6 Astra has the higher record proximity, 78.0 to 77.2 (where 100 means matching community records), though the 95% bootstrap interval for that gap includes zero. GPT-6 Astra also led all six hypervolume settings, which measure how much trade-off space each model's solutions cover.

Solving and optimising also proved to be different skills: Gemini 3.8 Flash solved 53 puzzles but found a new point on just one, while Fable 5.1 solved 52 and found new points on five.

How a record fell

The project's write-up traces Opus 5.5 on Undecimal Twin Rounder (SPEC44149538), a tournament puzzle that rounds an input to the nearest multiple of N on one output and of 11−N on the other. The frozen community record was 2,694 cycles. Opus 5.5's first passing program, 6.5 minutes in, took 5,669 cycles; it dipped under the record at the 16-minute mark and finished at 1,918 — 29% faster, at the cost of one extra node and 28 extra instructions. The core is a two-speed search: two nodes subtract 4N until overshooting, then add N back, and each drives a small coprocessor acting as a lookup table through jump offsets sent over a port.

The other cycle records: GPT-6.1 Sol reached 3,460 cycles on Intersection Identifier (SPEC64160260) against the community's 3,872, an 11% gain, and Gemini 4 Argon hit 188 on Error: 8 (SPEC79324720) against 198, 5% faster. All three records fell on tournament puzzles; none in the main game.

Cost and mechanics

Cost did not track quality. GPT-6.1 Sol reached 75.6 record proximity at an estimated $0.75 per puzzle, while Opus 5.5's 77.2 cost $5.08 — seven times the price for two points — and Fable 5.1 was the most expensive at $7.19 with a score of 70.2. The full run consumed 880 episodes, 95.5 million output tokens and 62,400 game-tool calls. Puzzle mechanics stratified the field: fifteen of the sixteen models solved all five routing puzzles, and none found a new frontier point there, while on multiplication puzzles, where the language has no MUL instruction, designs diverged — Gemini 4 Argon used a relative jump into a block of repeated instructions to multiply in 436 cycles, about 17% faster than GPT-6 Astra's 523.

Why it matters

Most coding benchmarks ask whether a model can write a correct program; this one measures sustained engineering — iterating under a clock, debugging from diagnostics and trading one resource against another. Every claimed result is an executable artefact verified on fresh inputs, which is hard to satisfy with memorised answers. And the loop closes with humans: within days of publication, community players had picked up the models' programs and improved on most of them, turning the leaderboard into a shared frontier between people and models. The authors say RL environments and traces are available to research teams on request.