Developer isolates CI jobs in Firecracker microVMs after malicious PR near-miss

A pull request aimed at the runner, not the code

A developer writing on dev.to has described a security near-miss that began with an unremarkable dependency-bump pull request to one of their open-source projects. Most of the diff looked routine, but one hunk quietly modified the CI configuration, adding a single extra step to every build. The author, Dhruv Malaviya, writes that they nearly merged it before reading the change properly.

The risk was not in the project's source code but in the machine that built it. The project ran on a self-hosted CI runner — a cheap VPS set up to save on CI minutes — that accepted jobs from repository forks and kept cloud credentials in a config file left there for convenience. According to the author, a malicious pull request would not have needed any sandbox escape: it could have posted the runner's environment to a pastebin and the runner would have complied.

The broader lesson, the author argues, is that a self-hosted runner is a machine that executes strangers' code and then keeps living. Their cheapest machine had become their most dangerous one.

Why containers weren't the answer

Containerising the runner is the usual response, and the author doesn't dismiss it, but lists several reasons it falls short on a CI host:

• The container shares a kernel with the host machine.
• Mounted caches outlive the job, so a poisoned cache is handed to the next build.
• Runners often expose the Docker socket.
• The filesystem persists between builds.

The author's summary is that containers on a runner amount to politeness rather than a boundary.

One disposable microVM per job

The fix described in the post is to give every CI job its own Firecracker microVM that is destroyed when the job ends. The setup runs on Krova Cloud, whose microVMs are branded Cubes; the author reports a fresh 2 vCPU, 4 GB RAM, 40 GB disk Ubuntu 24.04 machine booting in under a second.

The lifecycle is deliberately simple: the pipeline creates a Cube named after the build, injects secrets at runtime — a deploy token passed over SSH to the build script — and deletes the machine afterwards. The author calls teardown 'the security feature', since nothing persists afterwards: no poisoned cache, no leftover cron job, no credentials on disk for the next build. For orchestrators calling the provider's API directly, the post shows an idempotency key attached to the create request so a retried pipeline doesn't double-provision.

Firecracker, as the post notes, is the same virtualisation technology behind AWS Lambda, so a kernel exploit triggered inside a build lands in a machine that is seconds from deletion.

No address to attack

The job machines also have no public IP address. They sit on a private, NAT'd network with inbound traffic denied by default, and the author shows a clean listening-socket listing with nothing running that the job didn't start. A CI box that opens no inbound ports is not a scannable VPS with SSH exposed to the internet.

Cost, per the author, is per-minute billing: a 12-minute job on a 2 vCPU, 4 GB machine comes to a few cents, which is what makes an ephemeral-everything approach economically viable.

Isolation is not safety

The post is unusually blunt about the limits of the approach. A fresh VM shrinks the blast radius of a compromised build; it does nothing for credential hygiene. If a job is handed a long-lived production token, the author writes, the token simply gets stolen from a brand-new machine. On top of the microVM isolation they still recommend short-lived, scoped tokens per job, no shared volumes between jobs, and restricted egress where it matters. Anyone selling 'secure CI' as a checkbox is selling something, the post concludes.

Why it matters

Pull requests from strangers are the standard input a CI system is built to accept, which makes the pipeline itself a supply-chain attack surface: the code your build runs is code you didn't write. This account — a single, first-person report centred on one provider, and not independently verified — doubles as a practical checklist that transfers to any platform: treat runners as disposable, isolate each job at the hypervisor rather than the kernel, keep build machines off the public internet, and never let a long-lived secret touch a box that executes untrusted code. The near-miss here was caught by a careful read of a diff. Under the architecture described, the next one wouldn't have to be.