Brokk drops musl prebuilt option for Bifrost after 26% slowdown persists with mimalloc

The reversal

Jonathan Ellis is dropping musl from one of his Rust projects after benchmarking it against glibc and finding the lightweight C library to be far slower. Writing on the Brokk blog, in a post that reached the Hacker News front page, Ellis says the musl prebuilt option is being removed from Bifrost, a performance-sensitive project he maintains, while his smaller tools will stay on musl.

How musl got in

Ellis spent most of his career in the JVM with a sideline in Python, so when a containerized deployment of Bifrost ran into a libc incompatibility and an AI assistant recommended musl, he adopted it — and moved his other Rust projects over as well. The draw was statically linked binaries that depend on nothing on the host, shipped as a single artifact.

A colleague named Ryan then flagged musl's weak memory allocator and pointed Ellis to an article by Daniel Raneland covering the issue. Instead of taking the claim at face value, Ellis decided to measure it himself.

What the numbers show

The allocator problem is real, Ellis reports, and — contrary to other write-ups he had read — not limited to heavily concurrent workloads. His measurements come from 4-core EC2 VMs, with Bifrost's thread pools sized to that hardware, so the losses show up at ordinary levels of parallelism.

The standard remedy of swapping in a better allocator, mimalloc or jemalloc, falls short. In Ellis's tests, musl-linked builds running with mimalloc still came in 26% slower than glibc.

Two workloads explain why. One, called scan_usages, does allocate: mimalloc recovered part of the loss, but glibc still won. The other, structural_clone_smells, allocates very little — running it with or without mimalloc made no measurable difference — yet it regressed proportionally more than scan_usages did. That pushed Ellis to look beyond the allocator, and he concluded that a number of everyday memory routines in musl are slow in their own right.

A split decision

The findings did not send every project back to glibc. Ellis is keeping smaller efforts such as Hel on musl, now paired with mimalloc, on the grounds that a roughly 25% slowdown is tolerable where simplicity is the goal. Bifrost, where throughput matters, loses its musl prebuilt.

Ellis also argues musl should not bundle an allocator at all and should instead require developers to pick one. Opting for the slow choice because of a tiny footprint would then be a deliberate decision rather than a nasty surprise for anyone who missed the caveat.

Why it matters

musl is best known as the C library behind Alpine Linux, a common base for minimal container images, and static musl binaries are a popular way to ship Rust services as single files. Ellis's numbers are a concrete reminder that the size and portability gains carry a runtime cost — one that a better allocator only partly offsets, and that appears even in workloads which barely touch the heap.

For teams weighing a base image or a linker target, the practical lesson is to benchmark the real workload instead of assuming the tradeoffs, and to price image-size savings against throughput. For those already on musl with performance problems, mimalloc is a partial fix, not parity: in Ellis's measurements the gap stayed in the double digits even after the swap.