MLX v0.32.0 shipped wheels with no type stubs, and its tests never noticed

What happened

In July 2026, MLX published v0.32.0 wheels for macOS ARM64, and the build looked healthy on the surface: the wheel still carried py.typed, the marker file that tells static type checkers a package ships typing information. But according to a post on dev.to, the wheel contained no .pyi stub files at all — every stub that shipped with the previous release had quietly vanished from the package. Users who type-checked code against the installed library found their tooling broken, with nothing in the release flagging the loss.

The post identifies why the project's own test suite stayed green: tests run against the source tree, where the stubs still existed. The wheel is a separate artifact produced by a separate build step, and nothing in the standard publishing workflow verifies that what landed in the wheel matches what lives in the repository. twine check, the closest thing to a pre-upload gate in the common toolchain, only confirms that a README renders.

A repeated failure, not a one-off

The author points to a near-identical incident roughly a month earlier, when a built distribution from OpenSpace silently dropped its tracked host_skills/ directory, breaking integrations for everyone who installed it with pip. Both bugs share a root cause: nobody inspected the finished artifact. Build backends, MANIFEST.in mistakes and package-data misconfigurations can drop files at any step between repository, sdist and wheel — three distinct artifacts, as the post frames it — and the problem only becomes visible after installation, when a module or stub turns up missing.

wheeltruth inspects the wheel, not the repo

Fed up with learning about such breakage from users, the author built wheeltruth, a stdlib-only command-line tool that checks built distributions directly. Given a wheel, it verifies that the RECORD manifest is complete with matching sha256 hashes and sizes, that console and GUI script entry points resolve to modules actually inside the wheel, that stub files are consistent across a package — a partially dropped set, as with MLX, is flagged — and that the name and version in METADATA agree with the wheel filename.

Given both a wheel and an sdist, it diffs the two and reports files tracked in one but absent from the other, which covers the OpenSpace scenario. Two further modes exist for thoroughness: one validates packages declared in pyproject.toml or setup.cfg against the wheel's actual contents, and a smoke mode installs a wheel into a throwaway virtual environment and imports every top-level module. Exit codes are designed for CI — zero when clean, one when issues are found — so it can slot in as a verification step immediately after the build.

Limits and a scan of PyPI's top 300

The author is candid about v0.1's limits: it reports problems but does not repair build configuration, its expected-file heuristics can raise false positives on unusual layouts, and the stub check enforces only internal consistency, since the tool cannot know what a previous release shipped. Before publishing, the author ran the tool against its own distribution and got a clean result. A wider scan of the 300 most-downloaded PyPI wheels found zero hash mismatches, zero RECORD gaps and zero metadata mismatches — the top of the ecosystem held up, and the author declines to manufacture a broken-wheel statistic from it. The 45 flags that did appear were, on hand-checking, false positives that exposed three bugs in wheeltruth itself, involving vendored RECORD merging, entry-point re-exports and over-aggressive stub checks; fixes are planned for v0.2.

Why it matters

A green test suite says nothing about the artifact users actually install. When CI runs against the source tree and the release pipeline uploads a wheel without ever diffing it against the sdist or the repository, regressions ship silently — and for typed libraries, stubs are part of the public interface, so losing them is a breaking change even with every test passing. A post-build verification step, whether wheeltruth or an equivalent check, closes that gap for a few seconds of build time. It is also a decent example of tooling hygiene: the author tested the checker against its own output before trusting it on anyone else's, and then used a real-world scan to find the tool's own bugs.