POCKET-35B-GGUF runs a 35-billion-parameter LLM entirely on CPU

What was released

VIDRAFT, a Korean AI startup, has published POCKET-35B-GGUF, a large language model with 35 billion parameters packaged in GGUF — the weight format that succeeded GGML and is now the standard of the llama.cpp ecosystem. According to a dev.to write-up, the release targets machines with no graphics card at all: ordinary laptops and on-premise CPU servers.

The German AI publication Mindverse covered the release independently, which the dev.to author reads as a sign of developer interest extending beyond Korea.

The parameter count is the headline. Models distributed for CPU-first local inference have typically clustered at 7B or 13B parameters, so a 35B release meaningfully raises the ceiling of what a GPU-free setup can plausibly run.

How CPU inference at this scale works

The dev.to article lays out the mechanics. Post-training quantization lowers the numerical precision of the model's weights — for instance from 16-bit floats to 4- or 8-bit integers — shrinking RAM requirements by a factor of two to four or more while trying to preserve as much model quality as practical.

The file format does part of the work. GGUF bundles the weights together with the tokenizer, architecture configuration and quantization parameters in one file, which the article notes is far simpler to deploy than the older, fragmented checkpoint layouts.

On the runtime side, llama.cpp memory-maps the model file so that pages load on demand rather than requiring the entire model to sit in RAM before the first token is produced. Matrix operations run through CPU-optimised paths built on BLAS libraries and architecture-specific SIMD instructions, and layered offloading can spread work across CPU, RAM and swap where the runtime supports it.

The tradeoff is speed. CPU inference at 35B scale is slower than GPU inference, but the article argues that for batch jobs, offline document work, coding assistance and air-gapped deployments, cost per unit of throughput or data-sovereignty requirements can make the added latency worthwhile.

What the coverage does and does not say

No benchmark scores accompany the launch. The dev.to write-up is explicit that Mindverse's coverage focuses on the model's availability and CPU positioning rather than measured performance, and it declines to invent numbers where none were published. Anyone comparing POCKET-35B against alternatives will need independent benchmarks or the model card itself.

Memory requirements depend on the quantization variant. The article estimates that lower-bit options such as Q4_K_M could bring a 35B model under roughly 20-24 GB of RAM, with exact figures varying by variant.

Licensing is also unconfirmed. The write-up advises checking terms directly on VIDRAFT's Hugging Face model card rather than assuming permissive licensing from public availability alone.

Running it locally

Distribution is via Hugging Face in GGUF form, intended for llama.cpp-compatible runtimes. The general workflow is a huggingface-cli download, then either llama-cli for one-off prompts or llama-server to expose an OpenAI-compatible /v1/chat/completions endpoint that any OpenAI SDK client can call.

Because it is GGUF, the model should also work in popular local inference front ends. Both LM Studio and Ollama handle GGUF files and use llama.cpp underneath, so the downloaded file can generally be loaded directly, with the usual caveat to verify the specific quantization variant.

One caution the article stresses: the exact repository slug and GGUF filename should be confirmed from VIDRAFT's official model card rather than guessed.

Why it matters

Local deployment of capable models has been gated by two constraints: VRAM budgets and dependence on cloud APIs. A 35B model that runs on commodity CPU hardware widens the pool of organisations that can keep inference entirely on-premise — regulated industries, government bodies and anyone with air-gapped or data-sovereignty requirements who cannot ship prompts to a hosted endpoint.

It also narrows the gap between frontier-class and locally runnable models. If 35B on CPU becomes routine rather than exceptional, the default architecture for many internal tools shifts from remote API calls to a self-hosted server process, with latency accepted in exchange for privacy, control and predictable cost. The open questions — output quality at aggressive quantization levels, real-world throughput and licensing terms — are exactly what early adopters should test before committing.