Jane Street cut message bus CPU 30% with per-partition indexes and tree-splitting

A 30% CPU cut on a multi-terabyte-per-day bus

Jane Street has published an engineering deep dive on Aria, its internal messaging framework that processes multiple terabytes of data per day. According to the post, part of the firm's series on 2026 summer intern projects, intern Theodor Totev spent the summer replacing naive linear filtering with per-partition indexes and a tree-splitting storage strategy. The result was a 30% decrease in CPU usage on production workloads, achieved without lowering the correctness bar expected of a critical system.

Tip recovery was maxing out servers

Clients subscribe to Aria to receive a live stream of messages organized into topics, which form a hierarchical namespace similar to a file system. A subscription can target a single topic or an entire topic subtree. The most recent messages sit in an in-memory ring buffer called the stream tip; when a client falls behind, it re-reads recent messages from that buffer in a process Jane Street calls tip recovery.

Historically, filtering the stream down to a client's subscribed topics was a simple linear pass. It was algorithmically crude but cache friendly, so performance was acceptable — until the number of clients performing tip recovery grew. Some servers reached 100% CPU utilization, leaving clients unable to catch up with the live stream. Jane Street added more servers as a stopgap while recognizing that the recovery path needed a rethink.

Index topic partitions, not topics

The fix was to index the stream rather than scan it. A naive design would create one index per topic, but a single Aria instance can host almost a million topics. Totev instead built an index per topic partition: the set of topics sharing the same two-segment prefix, so app/codestore/commits and app/codestore/features both fall under the app/codestore partition. Each partition index records where its messages sit in the overall stream, and when a client requests data, Aria performs an n-way merge of the relevant indexes using a min-heap, reconstructing the ordered message stream for only the partitions that matter.

Cheap experiments: five heaps profiled overnight

Before trusting any of this, the team built a benchmarking tool to profile recovery scenarios, varying the number of topic partitions, the degree of interleaving and the number of readers. Profiling showed the min-heap merge was the biggest bottleneck. Jane Street notes that before AI tooling, engineers would probably have hand-picked a single promising replacement. Instead, Totev prompted an agent to produce five different heap implementations and profile them overnight; one of them, fast_heap_unboxed, delivered a 2x performance improvement to the indexed recovery code. The new logic was then covered with expect tests, exercised in Antithesis, and scrutinized by what the post describes as a swarm of agents.

A shared block pool for index memory

Memory representation mattered too. A dedicated ring buffer per index would need worst-case sizing, and index size grows as message size shrinks: with Aria's smallest 32-byte messages, a fully occupied tip store could require a 2GB index per partition. Instead, indexes draw from a shared pool of blocks holding 1024 entries each. Once every message referenced by a block has aged out of the ring buffer, the block is popped off the index and recycled, so each index grows and shrinks with its actual message volume.

Initial recovery: 2.5 seconds became 13 minutes

The post also covers a second, disk-side failure. Rebooting clients often request every message since the beginning of the week — potentially billions of messages — and reboots tend to cluster in time. In one incident, a recovery that normally finished in under 2.5 seconds took more than 13 minutes. The diagnosis found the same disease as the tip store: Aria was reading and filtering roughly 10x more data than it actually needed to send.

On disk, messages are first written as chronological segments per topic partition, then a separate process splits them into subtree stores that balance filtering efficiency against the cost of recombining files into a stream. The old splitting heuristic looked at the first three segments of a topic name — a segment being one slash-delimited part — which effectively gave each direct child of a topic partition its own subtree store. That failed when one topic inside a store carried far more traffic than another: a subscriber to the quiet topic still had to wade through its sibling's messages. The post frames the remedy as tree-splitting, the second half of the project credited with the overall CPU reduction.

Why it matters

The piece is a compact catalogue of transferable systems lessons. Filtering at read time quietly becomes read amplification, and both the in-memory and on-disk failures here came from the same waste pattern. Indexing at a coarser granularity — partitions rather than topics — keeps the index count tractable while still skipping most of the data. Benchmark-driven development caught a counter-intuitive bottleneck, and AI tooling changed the economics of experimentation: generating and profiling five implementations overnight, then re-verifying correctness with expect tests, Antithesis and agent review. The pooled block design shows memory layout being treated as a first-class design constraint. For anyone operating a message bus or log-structured store with many selective readers, the pattern — measure, index, re-verify — is the takeaway.