Four DGX Spark boxes serve DeepSeek to 16 users at once at 24 tokens per second each

What the test showed

An open-source serving engine built specifically for DeepSeek V4.1 Flash kept up with 16 simultaneous users on four linked Nvidia DGX Spark desktop machines, according to a report on dev.to. Aggregate throughput reached about 380 tokens per second, or roughly 24 tokens per second per person, on hardware that sits next to a desk rather than in a server room.

Single-user performance was higher still: 80 tokens per second in chat and 138 tokens per second during code generation. When the system processed 16 copies of the same request together, total throughput climbed to 438 tokens per second. The engine also ingested prompts at around 10,000 tokens per second and supported a context window of one million tokens.

For comparison, the custom build outperformed SGLang TP4 on the same four boxes by 18 to 25 percent in single-user tests, with prompt ingestion roughly 1.9 times faster, the report said.

Software was the constraint, not the hardware

A developer posting as Picomoorhead open-sourced the engine, which was written for this one model running on this particular four-unit configuration. Analyst Patrick Moorhead, founder of Moor Insights & Strategy, publicized the results and argued they answer the most common criticism of Nvidia's small machine. Nvidia has pitched DGX Spark as a desk-side system for developers to build and test models, leaving the job of serving many users to datacenter GPUs or cloud services.

"The knock on DGX Spark was that it's a developer box, not an inference box," Moorhead wrote, adding that this build suggests otherwise.

His conclusion was that the hardware was never the real limit; the software was. General-purpose serving frameworks must support many models across many hardware configurations, while an engine tuned for a single model on a single cluster can extract measurably more from the same silicon.

The physical rig is deliberately modest: four desktop boxes joined by a 200-gigabit network switch, with no liquid cooling. Moorhead described the combination of open DeepSeek weights, compact Nvidia hardware and community-written inference code as a workable template for departmental AI.

What comes next

The X post announcing the results said image support was on the way, while the project's GitHub documentation states that images already work through the standard OpenAI API, according to the report.

The open questions are repeatability and generalization. The next test is whether other teams can reproduce these numbers using the published code, and whether similarly model-specific serving engines start appearing for other open-weight models. The report also notes the obvious caveat: this engine is bespoke to one model on one hardware setup, so teams that swap in a different model or different machines would need comparable tuning work to see similar speeds.

Why it matters

Nvidia's own positioning left multi-user inference to the datacenter or the cloud, treating desktop hardware as a prototyping tool. Sixteen concurrent users on four desk-side boxes demonstrates a middle option: a small team can serve an open model locally, keeping sensitive prompts in house and capping cloud spending, without liquid cooling or rack infrastructure.

The result also reframes where the desktop-inference bottleneck actually sits. The gap between developer boxes and inference servers appears to have been a software problem, and a purpose-built open-source engine closed a meaningful part of it. If that pattern holds for other models, self-hosted AI for departments becomes less a research experiment and more a practical procurement choice.