Qwenputer compiles a working RISC-V CPU into the weights of Qwen 3.5

A developer has built a working 32-bit RISC-V processor inside a stock Qwen 3.5 4B model, with the CPU compiled directly into the network's weights rather than attached as an external tool. According to the write-up on divagr.com, which reached the front page of Hacker News, the project — dubbed Qwenputer — runs entirely within the model's forward passes and can execute arithmetic, C and MicroPython code with no sandbox, runtime or external tool calls of any kind.

Turning attention layers into RAM

The foundation is Qwen 3.5's architecture: three of its four attention layers use Gated DeltaNet, a linear RNN in which each head maintains a state matrix that the delta rule updates token by token. Normally that state decays, because a gate below 1 fades stored values at every step. The author found that pinning the gate to 1, setting write strength (beta) to 1, and encoding keys as one-hot vectors makes each head behave like addressable memory: a one-hot write overwrites exactly one row of the state, and a one-hot query retrieves an exact value. A GDN head's state also stays the same size no matter how long the conversation runs, unlike an attention KV cache, so this RAM is cheap.

Using the delta rule as associative memory is not new — Schlag, Irie and Schmidhuber showed in 2021 that linear transformers act as fast weight programmers. The novelty here is making the mechanism exact and addressable inside a production model, reliably enough to run a full CPU on top of it. Stock Qwen 3.5 does not keep exact values in its GDN layers; the modification unlocks that capability.

A CPU compiled, not trained

No training takes place. The processor is described as a circuit of small steps — read a register, add two values, jump if zero — and a compiler translates each step into transformer components: attention heads perform lookups, feed-forward units do arithmetic, and DeltaNet heads hold the memory. The CPU occupies 11 extra layers placed ahead of Qwen's 32. As the model decodes, it emits a trace of generated tokens per instruction — one naming the memory row to write, one carrying the value, and a STEP token closing the instruction — so the processor executes while Qwen generates.

The core implements the full RV32IM instruction set plus the Zba extension and a small bignum unit for big-number arithmetic. It is not yet a Linux-class chip. The author reports that it passes the official riscv-tests, matches qemu's output, and completed 18 million instructions free-running — continuing purely on its own outputs, with no reference machine correcting it between steps — without a single error.

How Qwen drives it, and the numbers

Qwen 3.5 4B uses the internal CPU with no post-training. The design piggybacks on the tool-call format language models are already trained on: Qwen writes instructions, the compiled layers execute them and write the response back, all internally. A small driver called RvChat loads the firmware into the GDN state. Qwen's original weights are left untouched, with the computer weights added in extra dimensions and layers.

The reported benchmarks hold up against a Python tool. On the DeepMind arithmetic sets, Qwenputer scores 97.5% on div_big versus 95% for base Qwen given a Python tool and 72.5% unaided; 97.5% on mul_big versus 100% with the tool and 55% base; 100% on mul_div_multiple_longer versus 97.5% and 90%; and 97.5% on add_or_sub_big, level with the tool. On NUPA long-integer arithmetic, it lifts the base model from 48.6% to 100%, matching a Python tool. About 230,000 output positions replayed against a reference machine showed zero mismatches, and general text behaviour stays within noise of the original model (KL divergence of 6e-4 against a 5e-4 noise floor).

A filesystem in the state

The build goes further with in-model file storage, exposed through internal put, append, get, list and delete operations rather than actual tool calls. File names are hashed into one of 1,024 directory slots in the DeltaNet state, with contents chained across rows at eight tokens per row. Capacity is 512 files, roughly 53,000 tokens of text. The state can be saved at the end of one conversation and reloaded into the next, and on questions about facts that changed between chats the model answers correctly 90% of the time.

Why it matters

This is a striking demonstration that an LLM's internal state can host deterministic, general-purpose computation, with sampling used only to drive it. If the claims hold up, compiled weights offer something external tools cannot: computation the model itself cannot bypass or tamper with, which the author's follow-up work explores as built-in monitoring instruments. The same property cuts both ways — another follow-up shows hybrid models can learn to hide reasoning in their RNN state and evade chain-of-thought monitoring. It also points at a practical path past the arithmetic limits of sampling: the CPU's outputs are exact, and replaying them against a reference machine showed no mismatches. The model is available on Hugging Face and ModelScope, and the code is on GitHub.