Vercel cuts CDN metadata lookup P99 latency 91% with sharded route metadata

Vercel says it cut P99 latency for CDN metadata lookups by 91% by changing how routing metadata is stored and fetched, moving from one cache entry per path to bounded groups of paths called shards. The CDN executes more than 80 million routing instructions per second on average, and the change also made deployments faster, according to a post on the Vercel blog.

From per-path fetches to shards

Every request hitting Vercel's CDN has to work out which target paths exist and how to serve them. The path in a URL often does not match the underlying resource: a request for /blog/hello-world might resolve to a dynamic route such as /blog/[slug], and the CDN may need to check several candidate paths before finding the right response. Bloom filters already rule out paths that definitely do not exist; everything that remains needs an exact metadata lookup.

Vercel previously stored that metadata as a separate object per target path, each fetched and cached independently. That worked for small projects that deployed rarely, but large deployments can contain hundreds of thousands of paths, and every deployment creates a fresh set of cache keys. Cache misses therefore became a recurring cost for large sites that ship often, and the old flow also required a dependent HEAD and GET per path.

The fix was to group metadata for many paths into shards, with each shard capped in size so any single lookup fetches a bounded amount of data. Fetching one shard warms the cache for every path inside it, so later lookups for those paths are served locally.

Fast fetches without parsing everything

Pulling in a shard only helps if the CDN does not have to decode the whole thing. Each shard embeds an index of fixed-width pointers marking where each entry starts; pointers are sized as whole numbers of six-bit Base64 characters so any one can be decoded in place without first parsing the index line as JSON. The routing process binary-searches the encoded paths using build-time byte offsets, taking O(log n) pointer reads and string comparisons, then parses just the matching metadata value on the following line. The rest of the shard is never parsed.

The layout reuses techniques from other Vercel systems: sorted, alternating key-value JSONL records borrowed from its Bulk Redirects dataset, and directly addressable Base64 structures originally built for the Bloom filters. The first metadata fetch for a deployment already identifies which shard holds a given path, so shard selection adds no extra round trip.

Finding the right shard size

Vercel initially tried multi-megabyte shards, betting that a small in-process LRU cache in front of a larger cache shared per region would keep them hot. In practice the regional cache hit rate was high but the LRU hit rate was low, because requests spread across many processes in each region, and transferring multi-megabyte shards cost more than expected. Very small shards had the opposite problem, making regional misses too common. Production testing landed on shards of roughly 200 KB, which Vercel says kept regional hit rates high while making LRU misses cheap to fill. The reported latency figures were measured on production traffic between August 5 and 12, 2026.

The team also evaluated shrinking shards further: front-coding the sorted paths, deduplicating metadata across JSONL documents, and a more compact custom serialization format. Offline simulations produced much smaller shards but predicted only modest latency gains, so the added encoding, compatibility, and rollout work was not judged worth it for this migration.

Rolling out safely

Because this lookup runs on requests to every deployment, a mismatch between the old and new formats could serve a stale route, a wrong status code, or a 404 for a path that exists. Vercel validated offline first with a harness that looked up every path in test deployments both ways and compared the answers. It then ran what it called shadow mode in production: behind a feature flag, a random sample of requests performed both lookups, served the old result, and compared the answers in the background over several weeks without slowing requests.

Differences were extremely rare. One turned out to be a bug in the old encoding, which packed paths into RFC 2047 encoded words and only surfaced when an emoji was split across two words. The new format stores paths as plain UTF-8, so that class of bug cannot recur.

Why it matters

This is a clear case study in the core edge-performance trade-off: batching data to cut round trips versus fetching more than a request needs. Vercel's numbers show how much headroom can exist in data layout alone, before reaching for heavier compression, which the company itself found not worth the complexity. The individual techniques, including bounded shards, inline indexes with in-place-decodable pointers, and shadow-mode validation before a routing change, are reusable patterns for anyone distributing routing tables or configuration to edge infrastructure at scale.