BITCOS pushes ternary LLM weights below the 1.58-bit barrier

A packing scheme slips under ternary's nominal floor

A preprint posted to arXiv proposes BITCOS, a new way to store the weights of ternary large language models that undercuts the roughly 1.585 bits per weight usually quoted as the format's information-theoretic floor. According to the paper, the layout stores weights more compactly than today's standard packing in 26 of the 29 models tested, dips to 1.485 bits per weight on the sparsest model, and raises end-to-end decode throughput by up to 1.18x on CPUs and 1.27x on GPUs. The submission, whose arXiv record lists Evangelos Georganas as the submitter and is dated 14 September 2026, was surfaced on the Hacker News front page.

Where the 1.58-bit figure comes from

Ternary LLMs quantize every weight to one of three levels: -1, 0 or +1. Encoding one of three equally likely symbols takes log2(3), about 1.585 bits, which is why the format is casually called "1.58-bit" and why that number looks like a hard lower bound. Real deployments do not even reach it: the paper notes that the prevailing format packs five ternary weights into a byte, and that with the power-of-two group sizes used in practice this works out to 1.625 bits per weight. Crucially, both figures assume the three symbols occur with equal frequency.

Measured models are mostly zeros

That assumption fails on real networks. The authors profiled 29 ternary LLMs and report that zeros can account for up to 51.5 percent of all weights. When roughly half the values are identical, most of the information needed to describe a weight is positional: where the nonzeros sit, and which sign each carries.

BITCOS: a bitmap plus compacted signs

BITCOS encodes exactly that. It keeps one dense bitmap recording, with a single bit per weight, whether that weight is nonzero, followed by a compacted list holding the sign of every surviving weight, again one bit each. Summed up, the layout costs 2 - z bits per weight, where z is the model's zero density. By that formula, any model whose zeros exceed 37.5 percent of weights is stored more cheaply than the 1.625 bits of five-trit packing. The paper says 26 of the 29 models measured cross that line, and the sparsest lands at 1.485 bits per weight, below the celebrated 1.585 figure.

Kernel and end-to-end results

A compact format only matters if silicon can decode it quickly. The authors describe tuned unpacking routines for AVX-512, AVX2 and Intel Xe2 GPUs, and report that at the zero densities real ternary models actually exhibit, the new layout delivers up to a 1.28x gain against production state-of-the-art ternary matrix-vector multiplication kernels. In end-to-end inference across five platforms — client and server CPUs, plus integrated and discrete Xe2 GPUs — decode throughput improved by up to 1.18x on processors and 1.27x on the GPUs.

Caveats worth noting

This is a version-one preprint and has not been peer reviewed. The GPU results cover Intel's Xe2 architecture only, so behavior on other vendors' hardware is untested here. And the benefit is density-dependent: three of the 29 models did not pack more tightly than the existing format, so gains shrink as a model's weight distribution approaches uniform.

Why it matters

Token generation in LLMs is memory-bandwidth bound, so every bit trimmed from a weight translates almost directly into effective bandwidth, throughput and power savings. Ternary models already exist to shrink memory footprints, and BITCOS squeezes further: down from 1.625 effective bits to as low as 1.485, with double-digit decode speedups. That matters most at the cheap end of the market — phones, laptops, integrated graphics and embedded devices — where memory is small and every watt counts.

The deeper lesson is that 1.585 bits was never a physical floor, only a floor for uniformly distributed symbols. Real weight statistics are skewed and exploitable, a principle that likely extends to other quantization schemes. And because BITCOS is a repacking of already-trained weights paired with optimized kernels, existing ternary models can in principle adopt it without retraining, which lowers the barrier to adoption considerably.