DSpark draft model claims more than 2x BF16 inference for Spark-X2.6-4B

What DSpark is

The team behind Spark-X2.6-4B has released DSpark, a 624-million-parameter draft model built to speed up the 4B model through speculative decoding. According to an announcement published on dev.to on October 31, 2026, average BF16 inference throughput improves by a factor of more than two with the draft enabled: 2.14x on an NVIDIA RTX 4090 running llama.cpp, and 2.50x on an Apple M5 Pro running MLX, averaged across the six datasets the team reports on.

DSpark is not a standalone chat model. It works as a companion to Spark-X2.6-4B-GGUF: the small draft generates up to seven candidate tokens per pass, and the 4B target checks all of them in a single parallel step. Confirming several tokens at once means the target takes fewer sequential decoding steps to produce the same text. The draft ships in BF16 GGUF format, and the target can run either BF16 or quantized GGUF weights.

The reported numbers

Target, device, runtime	Average speedup	Throughput, without → with DSpark
BF16, RTX 4090, llama.cpp	2.14x	95 → 203 tok/s
BF16, M5 Pro, MLX	2.50x	31 → 77 tok/s
Q4_K_M, RTX 4090, llama.cpp	1.98x	226 → 447 tok/s
INT4, M5 Pro, MLX	1.62x	71 → 115 tok/s

The benchmarks span AIME25, GSM8K, MATH500, HumanEval, MBPP and Arena-Hard-v2. The single best result in the set is a 2.94x speedup on MATH500 with BF16 on the M5 Pro, while the highest raw throughput is 532 tokens per second on GSM8K with Q4_K_M on the RTX 4090. As the announcement itself notes, these peaks come from two different test configurations.

The team also reports acceptance-length measurements, taken with seven draft tokens and a maximum output of 4,096 tokens: averages land between 4.0236 and 4.5522 across the four configurations. That count includes one extra token produced by the target itself, so the ceiling is eight, meaning a typical pass confirms just over half of the tokens theoretically available.

Caveats and practicalities

The dev.to post states plainly that these figures come from the published model card rather than a new independent benchmark, and that results vary with hardware, precision and workload. One visible pattern in the data: quantized targets gain less (1.62x to 1.98x) than BF16 targets (2.14x to 2.50x), plausibly because a quantized target already decodes quickly, leaving the draft's fixed overhead a larger share of each step.

On the integration side, the documented setup targets the XHToken fork of llama.cpp, with commands written for Linux, Bash and NVIDIA GPUs. Upstream llama.cpp support is described as in progress, and MLX results are documented separately. The weights are available on Hugging Face and ModelScope, and the Spark-X2.6 series is released under Apache 2.0.

Why it matters

Speculative decoding is one of the few levers that speeds up local LLM inference without shrinking the model, and the 4B class is already a sweet spot for consumer GPUs and Apple Silicon. If the reported gains hold on real workloads, DSpark moves Spark-X2.6-4B's responsiveness closer to that of a much smaller model while keeping the 4B target in charge of verification, so output quality should track the target rather than the draft.

The acceptance-length figures also hint at the limits. With roughly 4 to 4.5 of a possible eight tokens confirmed per pass, actual speedups will depend on how predictable each workload's text is — structured outputs may fare better than open-ended generation. Since every number here is vendor-published, the sensible move is the one the team itself invites: download the weights and measure on your own device, runtime and precision.