A weight-class guide to local LLMs: what fits your hardware and when cloud is cheaper

A September 2026 guide on dev.to takes an unusual approach to ranking open-weight models: instead of benchmark scores, it sorts them into four hardware tiers — from an 8 GB laptop to a multi-node GPU cluster — and then runs the arithmetic on when buying a machine beats paying for a cloud API. The author frames it as a dated snapshot, expecting half the figures to be out of date within a few months.

Laptop tier: 8 to 16 GB

At the bottom end, dev.to lists Phi-4-mini, around 3.5 GB in 4-bit quantization, which runs CPU-only on an 8 GB machine and handles logic and math tasks. Gemma 4 E4B at roughly 6 GB is called the strongest all-rounder for 16 GB machines, alongside Qwen3 8B for chat and documents and Qwen3 14B, which needs 12–16 GB of VRAM for coding work.

The guide's most practical warning applies here: model weights are not the same as total memory use. Gemma 4 E4B roughly doubles its footprint to about 12.5 GB when running the full 128K-token context, because the KV cache expands as a conversation gets longer. A model that looks like a tight fit on paper will collide with that limit partway through a task.

Workstation tier: 24 to 64 GB

This is the class the author singles out as the sweet spot. Qwen3.8-27B, a dense 27-billion-parameter model released under Apache 2.0, needs 16.5–19.5 GB for Q4_K_M weights plus roughly 1.5 GB of runtime overhead — small enough to sit entirely on a used RTX 3090 or 4090, with a 262K-token context window and native vision support. A 64 GB Mac mini M4 manages about 7 tokens per second, which the guide calls adequate for non-interactive work.

For anyone buying one machine for local inference, the advice is: 32 GB is the minimum, 48 GB is comfortable, and 64 GB removes the need to compromise on context length. gpt-oss-120b at roughly 65 GB in MXFP4 and the sparse Qwen3-Coder-Next 80B-A3B also land in this class.

One big machine: 96 to 512 GB of unified memory

Apple's August 25 Mac Studio refresh anchors this tier: an M5 Ultra with up to 512 GB of unified memory, starting at $5,499 for 96 GB, with another $4,000 for 256 GB and the 512 GB configuration shipping in late October. Sparse mixture-of-experts models suit this architecture, since they need capacity far more than datacenter-grade bandwidth.

Community-reported figures put DeepSeek V4 Flash, a 284-billion-parameter model with 13 billion active per token, at around 39 tokens per second on a 128 GB M5 Max. The guide cautions that most other numbers here are engine estimates, since the hardware has only just started shipping. Llama 4 Maverick 400B needs roughly 220 GB and delivers about 12 tokens per second on a 512 GB machine.

Rack-only models

Kimi K3, at 2.8 trillion parameters across 896 experts, is technically published and downloadable, but its MXFP4 weights span 1.56 TB. The realistic floor is about 40 GPUs for FP8 serving, and dev.to estimates an always-on 40-GPU deployment costs $60,000–115,000 per month in compute alone, before networking, storage and staff.

One structural lesson from this tier: although only about 104 billion parameters activate per token, all 2.8 trillion must sit in memory because the router can call on any expert at any moment. Sparse activation reduces the arithmetic per token, not the memory the model occupies.

Cloud pricing is inconsistent

The guide's API price table shows identical weights sold at very different rates: Kimi K3 spans a 2.35× gap between hosts, DeepSeek V4 Flash a 6.5× gap. Two providers charge exactly the same $1.40 in and $4.40 out per million tokens for GLM-5.3, yet one delivers 149 tokens per second while the other manages 29. Served context also often falls short of marketing: DeepInfra caps DeepSeek V4 Pro output at 16,384 tokens against an advertised 384,000.

Why it matters

Two conclusions stand out. First, the break-even math: $20,000 buys either a 512 GB Mac Studio or about 4.5 billion output tokens of GLM-5.3 at $4.40 per million. At 10 tokens per second, the machine would need more than fourteen years of nonstop generation to produce that volume. Break-even lands around 2–3 million tokens per day over a twelve-month horizon — a sustained pipeline few individuals or small teams actually have.

Second, the capability gap has collapsed. A dense 27B model that fits on a used gaming GPU now trails the 2.4-trillion-parameter leader by about four points on the BenchLM aggregate index the guide cites, at a time when Chinese open-weight vendors — Qwen alone has passed one billion downloads on Hugging Face — dominate the ecosystem. For most buyers, the question is no longer whether local models are good enough, but whether their token volume justifies the hardware.