PrismML shrinks Qwen3.8 27B to 5.9GB with ternary weights, claims 98.2% quality retention

What was released

PrismML, a startup focused on neural network compression that grew out of Caltech research, has released Ternary Bonsai 2 27B — a heavily compressed build of Qwen3.8 27B that the company says keeps almost all of the original's measured capability while being small enough to run locally. The announcement reached the front page of Hacker News on 17 September 2026.

The model takes text and image input, supports a 262K-token context window, and ships under the Apache 2.0 license, with weights available now. It succeeds the first Bonsai 27B, which PrismML released roughly two months earlier.

How the compression works

Instead of storing conventional floating-point weights, Bonsai 2 27B constrains each weight to one of three values — negative one, zero, or positive one — and layers FP16 scaling factors on top of groups of weights to recover precision. According to PrismML, this comes out to 1.76 effective bits per weight and a total footprint of 5.9GB, a bit more than one-ninth the size of the full-precision model. The technique is applied across the entire language model rather than to selected layers.

What PrismML claims

Across a benchmark suite covering reasoning, math, coding, instruction following, vision, and agentic tool use, PrismML reports a score of 83.9 for the compressed model, which it says amounts to 98.2% of the aggregate performance of full-precision Qwen3.8 27B. The previous Bonsai generation retained around 95%, so the new version closes most of the remaining gap — a level the company characterises as practically lossless.

PrismML emphasises where the capability is preserved, not just the headline number. Coding agents, tool-using systems, and long-running tasks are the workloads most sensitive to degradation, because small errors accumulate over many steps. The company argues the compressed model holds up in precisely these areas, and positions it favorably on capability per gigabyte against other low-bit alternatives that trade away coding, vision, or tool-use performance to reach a deployable size.

All of these figures are vendor-reported; the announcement points to a whitepaper for per-benchmark results.

Throughput and energy numbers

On an NVIDIA GeForce RTX 5090, the model reaches up to 143 tokens per second; on an M5 Max, 46.8 tokens per second. PrismML also measures 0.714 mWh per token on an RTX 4090, which it says makes the compressed 27B model 40% more energy-efficient than an 8B model running at full precision.

Platform support covers NVIDIA GPUs through CUDA and Apple's Mac, iPhone, and iPad through MLX, using custom kernels written for the low-bit format. PrismML is backed by Khosla Ventures, Cerberus, and Google, with continued support from Samsung.

Why it matters

If the retention claims hold up under independent evaluation, this release strengthens the case that aggressive low-bit compression is a deployment strategy rather than a compromise. A multimodal 27B-class model in under 6GB fits on consumer GPUs and Apple silicon that could never host the original, which changes which workloads can run without a cloud connection — private document analysis, coding-agent loops, and background assistants that stay local by default and escalate selectively.

The implications extend past local inference. Fitting larger models into the same memory envelope means serving more users per GPU in datacenters, and lower energy per token broadens what is viable on battery-constrained devices. As PrismML frames it, the useful question is shifting from raw capability to how much capability fits within a given memory, compute, and power budget.

The caveat is that the source here is the company's own launch post, with self-selected benchmarks and no third-party validation yet. The claim that compression at this level is effectively free is worth testing — but the trajectory from 95% to 98% retention in two months suggests the cost of running models locally keeps falling.