deniz.in

Markets

Weather

Loading weather

· via dev.to (home feed)

Tiny 2.1: a 367k-parameter QA transformer that runs from a floppy on DOS-era PCs

Tiny 2.1 is a 367,000-parameter question-answering transformer written in Turbo C 2.0 that fits on a floppy disk and runs on DOS-era PCs with 540 KB of memory.

Tiny 2.1: a 367k-parameter QA transformer that runs from a floppy on DOS-era PCs

A transformer that fits on a floppy

Tiny 2.1 is a question-answering transformer with roughly 367,000 parameters that fits on a single floppy disk and runs on PCs from the DOS era. According to Tom's Hardware, it is not a large language model but a small QA transformer that needs about 540 KB of memory and delivers its answers through the PC speaker. The project lives on GitHub as WollyMonjac/Tiny-2.1, was created on November 15, 2026, and ships under an MIT license for the code, while three derived data files keep their own licenses, documented in DATA_LICENSES.md. notebookcheck puts the entire package at around 435 KB, so it travels the way programs once did.

A character-level model written in Turbo C

The README describes a model with four layers and a width of 96 that reads and writes letter by letter at the character level. Weights are stored as one byte per value with a single scale per row, which keeps the parameter file to roughly 380 KB, and the program asks for about 540 KB of free conventional memory. A code inspection highlighted by ainewstodays adds finer detail: a 97-token vocabulary, a 128-character context window, four attention heads, a wide feed-forward middle layer and RMSNorm. The source is written in Turbo C 2.0 and rebuilds with a single TCC command; the README's target hardware is a Pentium or a DX-class machine running plain DOS.

Trained on 3,500 facts, built by AI

The training set is deliberately modest: about 3,500 facts, each phrased in many different wordings, and the README states openly that outside this set the model will confidently make things up. On questions worded the way they were taught it scores about 90 percent, and on 780 unseen test questions with new wordings and combinations it reaches about 61 percent, a large jump from the 6 percent managed by the first version.

One unusual detail is provenance. The README credits the C code to Claude Fable 5.2 and the training data to Opus 5.5, which notebookcheck identifies as Anthropic Claude models, meaning the whole pipeline from code to dataset was assembled by AI.

Speed and known limits

Response time depends on the host machine. With a math coprocessor, an answer arrives in a few seconds; without one, the coprocessor is emulated in software and an answer can take several minutes.

The README is equally frank about capability. The model struggles with questions about two countries at once, rare chemical elements and division, while its dependable arithmetic covers sums up to 20+20 and the twelve times table. It also has a habit of inventing an answer instead of admitting it does not know.

Attention has been steady since the release: Hackaday covered the project on November 18, and notebookcheck and MakeUseOf followed. A WASM port also runs the model in a browser, serving the original 380 KB TINY.BIN.

Why it matters

The project shows that a working neural network can run unassisted on hardware from the floppy era, which turns an old PC in a school lab, a museum or a demo setup into a working exhibit with no internet connection and no modern GPU. It also runs in a DOS emulator for anyone without the original hardware on hand. The repository includes the full training kit built on Python, NumPy and PyTorch, and an ordinary CPU is enough to retrain the model on a custom set of facts. For education, retro computing and demonstrations, Tiny 2.1 is a reminder that the interesting question is not only how large a model can grow, but how small a useful one can be made.

  • #ai
  • #retro-computing
  • #open-source
  • #ms-dos
  • #language-models

Related posts