LiteDRAM manual explains why DDR3 and DDR4 interfaces must be tuned at every boot

A hardware deep dive lands on Hacker News

A chapter from the manual of LiteDRAM, the open-source DRAM controller for FPGAs, reached the Hacker News front page on November 11. Hosted in the project's GitHub repository, the chapter lays out signal by signal what happens on the wires between an FPGA and its memory chips, and why a DDR3-or-later interface cannot simply be wired correctly once: every byte lane has to be measured and calibrated again at each boot.

The document works through the physical layer in order — the signal groups, the data strobes, the board routing, the leveling procedures, termination and reference voltages, then the FPGA's clocks and the DFI interface that joins the memory controller to the PHY. Each section ends by naming the part of LiteDRAM that implements what was just described, and a companion chapter enumerates the exact steps the BIOS carries out.

Two halves, many byte lanes

According to the manual, the interface splits into two asymmetric halves. Commands and addresses travel one way only, from controller to DRAM, and are shared by every chip of a rank: a differential clock pair, chip select, the RAS/CAS/WE lines that encode commands, address and bank pins, and control lines for power-down, on-die termination and reset. Data moves in both directions and is divided into byte lanes — eight data pins, a differential strobe pair and a data-mask bit apiece. Since every lane is routed independently, each one reaches the DRAM at a slightly different time, and each gets tuned individually.

Why the strobe, not the clock

Single-data-rate parts capture one word per clock on the rising edge of the controller's clock. DDR devices move two words per clock, and at those rates that clock can no longer be trusted for sampling: board traces and chip I/O introduce delays too large to ignore. The fix the manual describes is source synchronous — the transmitting side supplies a dedicated strobe, DQS, together with the data, and because both take the same path their delays cancel. Writes send the strobe centred on each bit; reads send it on the bit transitions and the receiver delays it by a quarter clock, with preamble and postamble framing the burst.

LiteDRAM arrives there by different means on different FPGAs: the Artix-7 PHY serializes its write strobe from a clock rotated 90 degrees, the Kintex-7, Virtex-7 and UltraScale PHYs use programmable output delays, and the Lattice and Gowin PHYs borrow the shifted write clock from their FPGAs' DQS blocks. On reads the approaches diverge further — the Xilinx PHYs ignore DQS and capture data with a fast internal clock through a tunable input delay, while Lattice and Gowin parts delay DQS by 90 degrees inside their DQS blocks and flag detected bursts.

Fly-by routing forces per-lane calibration

The core reason boot-time tuning exists is DDR3 and DDR4's fly-by topology. Clock, command and address pins form one bus daisy-chained past each chip and terminated at the end, which keeps those heavily loaded traces clean at high rates — but downstream chips latch the clock progressively later. The manual puts the skew between the first and last chip at potentially more than a full clock period, while the data lanes are point to point and inherit none of it. A strobe still has to land near each chip's own clock edge, so every lane needs its own delay.

Write leveling finds it: the controller sets a mode-register bit, pulses DQS, and the DRAM reports on DQ whether the strobe edge came before or after its clock edge; the delay is stepped until the sampled value flips. Later passes settle write latency in whole-clock increments, sweep DQ-versus-DQS timing to keep the middle of the passing range, and run read leveling, which pairs a coarse bitslip — which fast-clock cycle holds the first word — with a fine input delay of about 78 picoseconds per tap on 7-Series FPGAs with a 200 MHz reference, per the manual.

Read leveling and the data eye

On reads the controller cannot know in advance when data will show up, because board and I/O delays differ per lane. The resulting scan traces out the data eye through the delay line: generous margins indicate a solid interface, while a cramped one signals integrity or clocking faults. DDR3 and DDR4 can additionally return a fixed pattern from an internal multi-purpose register, letting the read path be tuned before anything has been written. PHYs that capture with DQS face an extra wrinkle — the strobe floats between bursts — so the FPGA gates it and reports burst detection.

Why it matters

Every system that boots DDR3 or DDR4 quietly depends on this calibration succeeding, yet the procedure is rarely documented outside vendor manuals. LiteDRAM makes each PHY declare which calibrations it supports, so the BIOS runs only what the silicon can do — including an Artix-7 workaround that shifts the clock itself, since that part lacks output delays. For engineers bringing up custom FPGA boards, writing firmware close to memory training, or chasing a design that works on one board and fails on another, the chapter is a rare public text that explains the physics and points at the code in the same place.