GPT-2 inference implemented entirely in CMake with Q16.16 integer math

A language model written as a build script

A project published on GitHub as AlpinDale/gpt2.cmake, which reached the front page of Hacker News, implements OpenAI's GPT-2 language model using nothing but CMake — the cross-platform build-configuration language normally used to set up C and C++ compilation. According to the repository's README, the model is executed with Q16.16 integer arithmetic, meaning all of the numerical work is done in fixed point rather than floating point.

What the repository provides

The project offers two ways to run it. The first is a small toy model: run tools/gen_tables.py and tools/gen_model.py, then execute the script with cmake -P gpt2.cmake. The second path targets the real checkpoint. The README instructs users to download model.safetensors, vocab. and merges.txt from the openai-community/gpt2 repository on Hugging Face, run tools/gen_full.py to prepare the data, and then start generation with a command such as cmake -P gpt2_full.cmake -DPROMPT="Hello" -DN=2.

The Python helpers appear only in the preparation step, handling the downloaded weights and tokenizer files. The generation itself is driven by CMake's script mode, where cmake -P executes a .cmake file as a standalone program instead of configuring a build. The code is released under a BSD 3-Clause license.

Why fixed-point arithmetic

The choice of Q16.16 follows directly from CMake's design. Its scripting language provides string variables and a math() command that operates on integers only; there is no native floating-point type. Q16.16 works around this by storing every number as an integer in which the low 16 bits represent the fractional part and the remaining bits the whole part — a classic technique from hardware without floating-point units. Since a transformer forward pass largely reduces to matrix multiplications, additions and a softmax over token logits, all of which can be approximated with scaled integers, the entire GPT-2 computation fits inside CMake's integer-only world.

GPT-2, released by OpenAI in 2019, is small by modern standards — the base checkpoint referenced in the README is the 124-million-parameter model — which is exactly what makes it the reference target for this kind of portability experiment.

Why it matters

The project is the latest demonstration that language-model inference is, at its core, portable arithmetic. Everything required — tensor operations, byte-pair tokenization, producing the next token — can be expressed in any runtime that offers loops and integer math. Similar stunts have targeted browsers, spreadsheets and SQL engines; doing it in CMake underlines how far a configuration language can be pushed past its intended purpose, and it runs anywhere cmake already exists, which describes an enormous number of development machines.

The fixed-point angle also has a serious counterpart: integer-only inference is the foundation of quantized models on microcontrollers and other edge hardware without floating-point support. Nobody should expect speed here — CMake scripts are interpreted and the tool was never designed for numerical work — but that is not the point. The point is that the boundary of where LLM inference can run keeps expanding, sometimes into places no framework vendor ever planned for.