SGLang's RadixAttention cuts multi-turn agent TTFT by 11x over vLLM, but only for shared-prefix traffic

What the benchmark shows

A dev.to analysis of inference engine benchmarks reports a large time-to-first-token (TTFT) win for SGLang over vLLM in multi-turn agent serving. In the headline scenario — an 8,000-token system context running Qwen2.5-Coder-32B-Instruct-AWQ, benchmarked by NextByteTech — SGLang's RadixAttention reduced TTFT from 440ms to 38ms, an 11.57x improvement. vLLM v0.6 with prefix caching achieved a 5.21x improvement in the same setting, going from 480ms to 92ms.

The post's actual argument, though, is that this number is conditional. Under a different traffic shape, the gap collapses or reverses, and the author assembled several published counter-cases to make that point.

Why RadixAttention suits agent traffic

Both engines reuse previously computed KV cache when requests share prompt prefixes. The difference is the data structure. According to the dev.to post, vLLM's automatic prefix caching hashes whole prompt blocks and matches them against cached blocks, which works when requests share long identical prefixes aligned to block boundaries but degrades when overlap diverges mid-sequence. SGLang's RadixAttention instead keeps the KV cache in a radix tree — a compressed trie over token sequences — so reuse can happen at the granularity of any common prefix, not just aligned blocks.

Multi-turn agent conversations fit that structure well: every turn repeats the same long system context plus a growing history, so nearly the entire prefill computation can be skipped on each subsequent turn.

A second dev.to comparison, by ifnodoraemon, illustrates the mechanism in the metrics. At a 4,096-token prompt with 75% prefix overlap and concurrency 64, SGLang reached a 78.6% KV cache hit rate versus vLLM's 41.2%. That difference showed up across the board: median TTFT of 85ms versus 380ms (4.47x), P99 TTFT of 280ms versus 1,250ms, and throughput of 5,430 versus 3,120 tokens per second.

Where the advantage evaporates

The post cites three counter-cases where SGLang's edge disappears:

• Unique prompts. A September 2026 bex.co report benchmarked Llama 3.3 70B FP8 on Spheron H100s at 50 concurrent requests with no shared prefixes. SGLang delivered 1,920 tok/s against vLLM's 1,850 tok/s, with TTFT within 20ms of each other — effectively a tie, since a radix tree has nothing to share when nothing repeats.
• Short-prompt batch throughput. The same bex.co report includes a May 2026 single-H100 run of Qwen2.5-7B where vLLM pushed 23,500 tok/s of raw throughput versus SGLang's 16,800 tok/s. SGLang still held a TTFT advantage (1.8s versus 7.8s), but for latency-tolerant batch work vLLM took the round.
• High concurrency. bex.co's DGX Spark sweep found SGLang ahead at 1–4 concurrent requests and vLLM ahead at 6 and above. Separately, RunPod's comparison on 2x H100 with DeepSeek-R1-Distill-Llama-70B at 7k context estimated a cache hit is worth roughly +20% speed, and found the two engines comparable on fresh context.

Questions to ask before switching engines

The post closes with a decision checklist: measure how much prefix overlap your traffic actually has; check your concurrency against the published crossover points, verifying on your own hardware; work out whether you are prefill-heavy or decode-heavy, since prefix caching only accelerates prefill and long outputs spend most of their time in decode; and identify which latency percentile your SLO targets, because SGLang's reported edge is concentrated in TTFT and tail latency rather than aggregate tokens per second.

Why it matters

Agent workloads are becoming a dominant pattern in production LLM serving, and TTFT directly shapes how responsive an agent loop feels to users. The cited numbers suggest that for this traffic class, cache architecture — not raw compute — is the deciding factor, and a radix tree's finer-grained prefix reuse is a genuine structural advantage over block-based hashing.

The same data is also a caution against benchmark-driven engine selection. Unique-prompt serving, short-prompt batch jobs and high-concurrency chatbot traffic all show the engines tied or vLLM ahead, per bex.co, RunPod and the second dev.to comparison. Teams should profile their own prefix overlap and concurrency before migrating. One further caveat: all figures here come from a single dev.to post citing third-party benchmarks, and the vLLM results reflect v0.6, so independent verification on target hardware is warranted.