containerd 2.2's mount manager panics on one-mount Activate chains

A one-call provisioning path

containerd 2.2 shipped a mount manager: an embeddable service that can create a file, format it as ext4 or xfs, attach it as a loopback device and hand the result to a runtime, replacing the usual truncate, mkfs, losetup and mount sequence with a single Activate call. A developer who tested it hands-on and published the findings on dev.to reports that the component works for well-formed requests, but a one-mount activation crashes it outright, some errors come back with the wrong type or no wrapping at all, and on the author's measurements the new call is slower than the shell commands it replaces.

There is no ctr subcommand for any of this. The manager lives in containerd's core mount package and is meant to be embedded by a snapshotter or a runtime shim. Activate returns two groups of mounts: the ones the manager handled itself, such as the loopback attach, and the ones the caller must still mount with the ordinary mount(2) syscall. The author tested against the containerd bundled with Docker Engine 29.3.1 (reporting itself as 2.2.2), pinning the package version in go.mod rather than trusting the daemon's version string.

Slower than doing it by hand

Both paths in the test format a 200MiB ext4 image, attach a loop device and mount it. The manager's Activate took 29.6, 39.0 and 47.2 milliseconds across three runs; the manual four-command sequence took 24.4, 23.6 and 25.9 milliseconds. Reading the transformer source, the author found it simply shells out to the real mkfs.ext4 and mkfs.xfs binaries, so the manager adds bookkeeping, including BoltDB writes and symlink creation, rather than removing work. Teardown showed the same gap: Deactivate plus umount cost 32 to 45 milliseconds, against 11 to 13 milliseconds for umount plus detaching the loop device by hand.

The report also documents a benchmarking pitfall the author hit along the way. An initial comparison used dd to pre-create the image, which writes real zero bytes and took 358 milliseconds to 1.6 seconds, making the manager look ten to fifty times faster than it is. Once the manual path used truncate, which creates a sparse file in under a millisecond, that apparent win disappeared and mildly reversed. Disk usage was identical either way: 200MB apparent size and about 17MB of actual blocks.

Errors that mislead

Invalid inputs are mostly rejected cleanly. An unsupported filesystem type such as btrfs and a missing size option both fail with "invalid argument" before any file is created. Two cases are rougher:

• A source path outside the manager's configured root is rejected as "not implemented", the same error class containerd uses for features that genuinely do not exist. Code that checks for a not-implemented error to decide whether to fall back to a different mount path would treat a misconfigured directory as a missing feature.
• Activating a second time under the same name, without deactivating the first, surfaces a raw bbolt "bucket already exists" error straight from the metadata store, with no containerd-level wrapping. It is accurate but describes the storage engine rather than the caller's mistake.

The panic

The documented examples always chain at least two mounts: one that produces a loop device and one that consumes it. Activating a single mkfs/loop mount on its own, with nothing consuming its output, panics with an index-out-of-range error inside Activate, at manager.go line 421 in the reported stack trace. The author traced the cause in the source: the code tracks the index of the first mount the caller is expected to handle, and for a mount that only ever serves as a transform target that index ends up equal to the length of the mount list, which a later loop then uses to index into the mount slice. Alongside the crash, the report notes the run left a loop device attached that nothing could subsequently find.

Concurrency holds up

Ten goroutines calling Activate in parallel against the same manager instance, each formatting its own 50MiB image, all succeeded in 70.1 milliseconds of wall time, with individual calls between 30.5 and 69.8 milliseconds. The author reasons that serialized activations would have taken several hundred milliseconds, so the manager's internal locking does allow genuinely concurrent formatting rather than queuing callers.

Why it matters

The mount manager is a new API surface aimed at snapshotter and shim authors, and it is positioned around composability rather than raw speed, a distinction this benchmark makes concrete. Anyone adopting the Activate-based provisioning path in containerd 2.2 should know its current edges: a single-mount chain is a hard crash rather than a validation error, error types can misdirect fallback logic, and duplicate activations leak storage-engine internals. Defensive callers can mitigate today by always chaining a consumer mount after a producing one, validating configured roots before delegating to the manager, and treating the unwrapped bbolt error as a sign the name is already active, but the panic and the error classification look like issues for upstream to fix before the new path is safe to treat as a drop-in replacement.