Copy-and-patch JIT: pgrust author details a compiler that emits code in 5 microseconds

A compiler fast enough for every query

A blog post on malisper.me that reached the Hacker News front page makes a pointed claim: just-in-time compilation that finishes in microseconds is now practical to build by hand, and databases that exploit this can outperform older engines. The author is building pgrust, a database whose custom JIT, according to the post, turns code into executable machine code in roughly 5 microseconds — little enough that every incoming SQL query gets compiled, not just a hand-picked subset.

The claim, and the gap it targets

The post opens with an observation about the industry: no production-ready database today ships a JIT compiler of its own. Engines that JIT at all either link against LLVM or emit C or C++ and invoke a compiler on that, and both routes carry compile times long enough to restrict where the technique is worth applying.

The author's argument is that this gap persists because writing a compiler that emits assembly directly used to demand rare, low-level expertise. Building pgrust's JIT, the author expected the same difficulty and instead found the task far more tractable, crediting AI assistance throughout. The resulting compiler is described as one of the main reasons pgrust performs well.

Why JIT at all

Per the post, JIT means generating compiled code while the program runs. When it works, the gains run from roughly 2x to 5x and occasionally beyond. The situation that justifies it is information arriving at runtime that drastically changes what the program should do. Interpreters are the canonical case, since they only receive the code to execute at runtime; the post also points at parsing, where the schema of the incoming data may not be known until execution.

The worked example: a regex engine

Rather than SQL, the tutorial builds a minimal regular expression engine. It supports exactly two features, literal strings and Kleene-star repetition, so it can match patterns such as b(an)* but nothing with alternation or lookaround. The regex arrives as an already-parsed Rust AST with three node types: Literal, Concatenation and Repetition.

The interpreted version fits in fewer than 20 lines of Rust. Benchmarked against a hand-written matcher specialized for b(an)*, the author reports the hand-written code runs 10 to 20 times faster across a couple of test examples — the gap the JIT is meant to close while staying fully general.

Copy-and-patch on ARM64

The technique the post teaches is a variant of copy-and-patch. The compiler holds a set of assembly templates, called stencils, one per operation it needs to emit. To compile an operation, it patches the relevant stencil with the operation's specifics and appends it to the output; chained together, the patched stencils form a program whose performance approaches the hand-written version. A final step copies the finished instructions into executable memory so Rust can call them like an ordinary function.

The walkthrough targets ARM64 on macOS and works backwards from the generated code. Notable design choices include an explicit stack for backtracking, and terminating the input with a null byte so character comparisons simply fail at end-of-string, removing the need for any length checks. Registers get fixed roles: x0 tracks the current input position and doubles as the return value, x1 and x2 hold the top and bottom of the backtracking stack (the bottom register makes emptiness detectable), and x9 serves as scratch space. The prologue is a single instruction initializing stack state; a character match compiles to a byte load, a compare, a conditional branch to a fallback block, and a pointer advance. From there the post moves into the repetition and backtracking logic, the emitter that fills stencils from the AST, and the packaging of generated code into a callable function.

Why it matters

For database and systems engineers, compile latency has always been the tax on JITting. LLVM-backed compilation inside a query engine only pays for itself on hot or expensive queries, which is why engines gate it behind thresholds and warm-up periods. A compiler measured in microseconds inverts that economics: per-query compilation becomes cheap enough that the gating logic can simply be deleted.

The secondary point concerns who can build such systems. The author's experience suggests AI assistance has lowered the barrier to assembly-level compiler work that previously kept databases dependent on LLVM, and frames this as an opening for new engines to differentiate against incumbents. The usual caveats apply: the numbers are the author's own, the benchmark is a toy with two regex features, and the post doubles as promotion for pgrust. Even so, the technique it documents is self-contained, and replicable by any team facing the same trade-off between compilation cost and runtime speed.