deniz.in

Markets

Weather

Loading weather

· via Hacker News – Front Page (native)

Coding agents take a developer with no FPGA experience to 213 tokens per second LLM inference

A developer with no Verilog or FPGA experience used coding agents to run a ternary-weight LLM on AWS FPGA instances, speeding it up 33x to 213 tokens per second for about $10,000 in tokens.

Coding agents take a developer with no FPGA experience to 213 tokens per second LLM inference

What happened

A developer with no FPGA experience has used AI coding agents to get a large language model running on reprogrammable hardware, pushing it to 213 tokens per second — about 33 times the speed of the first working version. The author recounts the project in a blog post on ankitg.me that reached the front page of Hacker News on November 20, 2026.

According to the post, the author had never written Verilog, the language used to describe circuits on FPGAs, and had not written any serious C++ in roughly a decade. Coding agents did essentially all of the low-level work.

The setup

The work ran on AWS F2 instances, which expose FPGA hardware in the cloud along with a relatively limited 16 GB of FPGA HBM. Getting started was itself delegated: the author asked Codex to reserve the instances. For the model, the author chose a ternary weight architecture from DeepGrove, a design well suited to memory-limited hardware.

A cloud agent, orchestrated through Conductor, then ran for several days across hundreds of experiments. The session produced roughly 3 million lines across the experiment history and involved parsing 30 million lines of output files, at a cost the author puts at about $10,000 in Claude and OpenAI tokens.

The human contribution was thin by any standard. Prompts reportedly consisted mostly of encouragement — the author cites examples like telling the agent to be more creative — with occasional ML nudges such as suggesting speculative decoding or asking why a component was still running on the CPU. Two models, referred to in the post as Astra and Opus, were used interchangeably, and both proved capable enough to alternate between.

From working to fast

The first full-model, bit-exact baseline ran at 6 tokens per second. The agent then climbed a measured optimization sequence to 213, and the individual steps mix standard performance engineering with hardware-specific tricks. One step combined a head loader that reads both of its memory banks at once — a measured 17.3 percent gain — with a runtime expert-prefetch policy and a router timing fix, and lowered the clock by one step to 225/425 MHz after a soak test surfaced random faults at 231.25 MHz.

The most substantial human input came from the DeepGrove founders. When the author reported roughly 10 tokens per second, they were surprised it was working so easily, saying they had attempted something similar a few months earlier and the models were not up to it. They supplied a prompt summarizing lessons from a decode-optimized FPGA implementation of a dense, Qwen-style model with ternary weights. It covered multiplier-free ternary math, where a 2-bit weight selects between a value, its negation and zero; 2-bit weight packing to cut memory transfers; weights streaming straight from HBM into the compute lanes; an MXINT8 KV cache; and attention processed in tiles with an online softmax so the full score matrix never has to be stored. The author pasted it into the session and calls it the only significant piece of real guidance added.

Why it matters

This is one anecdote, not a benchmark, but it isolates something concrete about the current moment: coding agents can open up domains where the barrier is accumulated specialist knowledge rather than raw intelligence or compute. FPGA development has traditionally meant hardware engineers, C++ toolchains and manual analysis of simulation waveforms. A software developer with a large token budget — the author credits YC for that — and a well-chosen problem got a working inference engine fast enough to matter in real time.

Two details keep it from being a pure stunt. First, the DeepGrove founders' failed attempt from a few months earlier suggests the capability threshold was crossed recently, and quickly. Second, the expert prompt mattered: agentic persistence got the model running, but distilled domain expertise shaped how fast it ultimately went. The most useful pattern here may be the pairing — specialists writing their knowledge into prompts, and generalists pointing agents at problems that used to be locked away.

  • #coding-agents
  • #fpga
  • #llm-inference
  • #hardware
  • #aws

Related posts