Cloudflare halves DNS build delay and cuts peak memory 85% with targeted zone builds

What Cloudflare changed

Cloudflare has reworked the Zone Builder at the center of its DNS propagation pipeline so that zone-setting changes no longer force a rebuild of an entire zone. According to the Cloudflare blog, the result is an 85% reduction in peak memory use and half the time between a customer's DNS change and the resulting data being written to Quicksilver, the distributed key-value store that carries DNS data across Cloudflare's network.

The pressure behind the work is growth. In 2022 the Zone Builder received about 250 DNS record changes per second; today it receives roughly 700, plus around 100 other events such as zone setting changes. Cloudflare says event volume was never the real issue — the problem was that certain events require work that scales with zone size, and some zones now hold more than 20 million records, a figure the company expects to climb toward hundreds of millions. Cloudflare's earlier per-record targeting already showed the payoff: in 2022 those builds were 150 times faster than full builds on average, and 4,250 times faster on a test zone with one million records. The open question was doing the same for zone settings.

How DNS changes reach the edge

Cloudflare describes a multi-stage pipeline. A customer edits a record through the DNS Records API or dashboard, or another Cloudflare service adjusts a zone setting. The change lands in a database, which produces a Kafka event. A DNS Kafka Aggregator combines events from several topics and passes them to the Zone Builder, which reads the needed zone data, applies DNS serving rules, converts the result into keys its authoritative DNS servers understand, diffs them against Quicksilver, and writes only what changed.

Why full builds stopped scaling

The original builder had one virtue: it did not need to know why a zone changed. It read the zone and its settings, encoded every record, compared against Quicksilver, removed stale keys and wrote updated ones. Although the final write was already minimal, the process still read, encoded and held every record in memory along the way — and for the largest zones, the whole zone does not fit in memory at all.

Zone settings made targeting harder. Cloudflare points to Universal SSL: toggling it updates metadata and can alter CAA records at the zone apex. A targeted build therefore has to answer which DNS names can change, whether shared metadata or the published-zone list can change, and what old state must be removed from Quicksilver.

Choosing the smallest safe build

Cloudflare reports solving five problems: making events state what changed, mapping each change to the smallest correct build, accounting for data shared by zones with the same name, coping with a database that can shift between event and build, and combining work without losing Kafka messages.

Event producers now include changed zone fields and settings, with previous and new values where available, and Cloudflare traced each setting through the DNS output code to classify its effects. Ambiguity still defaults to the expensive path: timestamp-only updates, unknown events, or changes that might affect records anywhere in the zone still trigger a full build. Uncertainty always means more work, never less.

The planner picks among five build types: resources only (the usual path for A, AAAA, TXT and CNAME changes), resources plus shared metadata (nameserver and wildcard changes), apex plus metadata (settings such as Universal SSL), a purge for zones that should stop being published, and a full zone rebuild when nothing smaller can be proven safe. The safety rule is strict — every key a targeted build writes must match, byte for byte, what a full build would produce, or the builder falls back to the full build.

There is one deliberate exception. Above a fixed record-count threshold the builder never escalates to a full build, because a full build of those zones cannot complete and, while retrying, blocks every other change on its Kafka partition. Instead it rebuilds the apex, the metadata and every name named by queued events, with zone holds limiting the remaining risk.

Why it matters

For infrastructure practitioners, this is a case study in scaling an event-driven pipeline by shrinking work to what can be proven necessary. The byte-for-byte equivalence invariant is what makes the optimization safe, and the memory headroom it created bought a second win: Cloudflare could safely shorten resilience waits the pipeline had always relied on, which is what halved end-to-end delay. The design also shows how to handle uncertainty honestly — falling back to full builds rather than guessing — and how to accept a pragmatic exception when a zone is simply too large to rebuild at all.