PrismML shrinks a 27B-parameter model to 5.9 GB in bid for on-device AI

A 27B model in under 6 GB

PrismML, an AI research lab founded by a group of Caltech researchers, released its newest compressed language model, Bonsai 2 27B, on Thursday. According to TechCrunch, the model squeezes Qwen3.8 27B, a widely used open-source model from Alibaba, down to 5.9 GB — a 9x to 10x memory reduction that makes it small enough to run on a PC and possibly a high-end smartphone.

The company is led by Babak Hassibi, a Caltech professor who specializes in compression technology, and it has raised a $22.25 million seed round from Khosla Ventures, Cerberus Capital and Caltech. Ion Stoica, co-founder of Databricks and director of Berkeley's Sky Computing Lab, sits on the startup's side as an adviser.

Benchmarks nearly unchanged

What sets the company apart, Hassibi told TechCrunch, is how little performance the compression costs. Bonsai 2 reportedly matches 98% of Qwen's aggregate benchmark scores, an improvement over the 95% retained by the original Bonsai, which was released in March. The company says that first model has since been downloaded more than 11 million times, with its smaller models adding another 2.6 million downloads.

Perfect parity may never arrive: Hassibi expects compression to always carry at least some cost. TechCrunch also argues the remaining gap is largely academic, since even uncompressed models are far from perfectly accurate, benchmarks only loosely reflect real tasks, and the surrounding software stack influences results as much as the model itself.

Ternary weights do the shrinking

The technique works by changing how model weights are stored. Weights are the values a model learns during training, and they are normally stored at 16 bits each. PrismML instead uses what it calls ternary weights, which can only hold three values: +1, −1 or 0. Restricting every weight to such small values cuts the model's storage footprint dramatically.

The next target is scale. Hassibi told TechCrunch that upcoming releases, hoped for within the next couple of months, will land in the several-hundred-billion-parameter range, and he expects intelligence to be easier to preserve there. His argument is that bigger models have more room for compression, so closing the final performance gap becomes simpler as size grows.

PrismML is not the only player here. TechCrunch cites Multiverse Computing, founded by a well-known professor at Spain's Donostia International Physics Center, as another compression-focused firm — one that has raised substantially more capital. A rumor that PrismML is in talks with Apple also appears in the report; Hassibi declined to comment on it.

Why it matters

Capable reasoning models today mostly live in data centers, which means API fees, inference costs, network latency and the privacy trade-off of shipping user data to the cloud. If PrismML's compression holds up at larger scales, that equation changes: models with hundreds of billions of parameters could plausibly ship inside hardware people already own. Stoica frames the appeal in exactly those terms, telling TechCrunch that on-device models make intelligence effectively free to run and private by default, since nothing has to leave the device. The download figures suggest real appetite as well — more than 13 million combined downloads for a company that, until its seed round, was barely known. For the wider industry, credible near-lossless compression would pressure API-centric business models and shift where AI workloads actually run, from hyperscaler clusters to laptops and phones.