Anthropic's CL-Bench shows memory lifting Claude Opus 5.5 from 0.655 to 0.838

Anthropic puts numbers on agent memory

Anthropic has quantified what persistent memory is worth to its coding agents. In an engineering post titled "The components of agent memory," written by engineer Lance Martin, the company reports Continual Learning Bench (CL-Bench) 1.0 results from 6 November 2026: Claude Opus 5.5 scored 0.838 when allowed to keep notes between sessions, against 0.655 without them. A dev.to write-up covering the post walks through both the numbers and the architecture behind them.

The post defines memory narrowly: information picked up from interactions or tasks, stored as readable text and reloaded later. Model weights, learning inside a single conversation, and compaction summaries are excluded. Only what survives between sessions as text counts.

How the benchmark works

CL-Bench is a database task. The agent answers questions, the environment reveals the correct result after each answer, and every question starts in a fresh session. Anything the agent carries from one question to the next has to travel through its memory files.

Model	Without memory	With memory
Claude Opus 5.5	0.655	0.838
Claude Sonnet 5.5	0.644	0.833
Claude Opus 4.7	0.383	0.700
Claude Sonnet 5	0.306	0.610

Two patterns stand out in the dev.to summary. Older models gained the most — Opus 4.7 nearly doubled — which suggests memory substitutes for reasoning strength by supplying facts the model would otherwise have to re-derive. And memory is not automatic: Sonnet 4.6 barely improved until Anthropic sharpened the instructions explaining what memory is for, after which it reached 0.641. The stated rule is to describe the purpose of memory without scripting exactly how to use it.

Four layers, one hierarchy

The architecture stacks four layers, each with a single job:

• Transcripts: the raw record of conversations and activity. Complete and searchable, but retrieval tends to surface the first match rather than the correct one.
• Memory notes: short lessons, rules and plans the agent writes itself. Compact, but only as reliable as the judgement that created them, and a wrong note stays wrong until something checks it.
• Memory representations: organised structures such as a linked wiki, giving the agent a map over a growing body of knowledge.
• Injected context: a small block loaded into every session. It must stay small because it spends tokens everywhere; Claude Code loads only the first 200 lines or 25 KB of its MEMORY.md index at startup.

Dreaming between sessions

A maintenance pass Anthropic calls dreaming runs while the agent is idle. It re-reads past transcripts alongside current memory, rewrites incorrect notes, merges duplicates, adds links and trims the always-loaded block. The Dreams API runs this as an asynchronous job over one to 100 session transcripts. The idea echoes sleep-time compute, proposed by researchers at Letta and UC Berkeley in April 2025, in which models use quiet periods to reorganise what they already know.

The clearest demonstration came from Pokémon Red. Sonnet 3.7 wrote 413 memory files across 225 folders, and one recorded the wrong side of the map for a PokéCenter near Mt. Moon — sending the agent off a one-way ledge into a loop it could not escape. A dreaming pass compared the note with the transcript, caught the error and rewrote it. With the fix, the agent reached Cerulean City at action 1,935; without it, the run never got past the mountain.

A wiki over 350,000 files

Anthropic also built a wiki over roughly 350,000 internal files from Stainless, a company it acquired in May 2026 — about 420 million tokens in total — using 128 sequential Opus 5 sessions and a 25-question evaluation. As dreaming expanded the wiki from two sessions to 128, performance rose from 13% to 48% of graded criteria. Search plus the wiki reached 61% against 53% for search alone, while saving about 20,000 tokens per question. On questions needing synthesis, judgement or arithmetic, wiki-based answers scored 87% versus 73%. The gains concentrated on questions spanning several documents; single-document tasks showed no clear benefit.

The dev.to author also built a working version of all four layers in roughly 100 lines of standard-library Python, including a dreaming pass that catches a wrong port number and a duplicate note in a scripted three-session scenario — evidence that the pattern needs little machinery to reproduce.

Why it matters

Agent memory is usually discussed as a vague benefit — helpful, probably, but unmeasured. Anthropic's numbers give it a shape: double-digit gains on a continual-learning task, the largest lifts for older and cheaper models, and real token savings when knowledge is organised into a wiki rather than retrieved by search alone. The results also carry warnings. A single bad note can derail every later session, injected context taxes every conversation, and models will ignore memory unless instructions explain its purpose. For teams building coding agents, the takeaway is that memory is an engineered component with its own maintenance loop, not a checkbox.