GPT-6 Astra's KiCad demo puts AI circuit design to the EEBench test

OpenAI puts circuit design on the launch page

OpenAI placed a demo of GPT-6 Astra working on a circuit board in KiCad on the front page of its launch post, a placement the team behind the EEBench benchmark found genuinely exciting — electronics rarely gets this kind of billing in a major model release. But as they write in a post titled "Can AI design circuit boards yet?", the demo also raises a question they have been circling for a while: how do you measure whether the electronics an AI produces are actually any good? Their answer is a benchmark that grades designs through simulation rather than by how skillfully a model drives a CAD application.

GUIs are the wrong interface for agents

According to EEBench, current models know far more about electronics than their output in conventional tools suggests, having absorbed textbooks, datasheets, application notes and large amounts of code. An agent operating a graphical CAD package burns much of its effort clicking through menus and tracking what is on screen, with its context dominated by coordinates and application state.

EEBench builds on atopile instead, where the circuit is expressed as declarative code. The agent manipulates parts, nets and electrical constraints directly, and can revise a design, build it, simulate it and inspect failures without leaving the project. The team says this has worked far better than asking a model to draw lines in a GUI, and it shifts the benchmark's attention from computer use to electronics itself.

Nominal values versus parts that arrive

One public task is based on a residential energy meter: when the 5 V supply drops out, the circuit must hold the processor up for a further 20 ms so it can save its accumulated reading, keeping the protected rail above the processor's 3.0 V brownout threshold throughout. EEBench says most models intuitively reach the right first answer — add a capacitor — but real components complicate things. A ceramic part can deliver much less than its advertised capacitance once voltage sits across it, tolerances bite, and extra capacitance costs money, occupies board space and slows rail recharge when power returns. A design that passes with nominal values can fail with the parts that actually ship.

The harness cuts input power in simulation and measures the rail through the outage, the effective capacitance at the operating point, recovery after power returns, and limits on package, dielectric, voltage rating and cost. Harder tasks go further: an agent may need to synthesize a multiple-feedback low-pass filter around an op-amp, solve the resistor and capacitor ratios for the required poles, and keep gain, cutoff frequency and Q within limits as every component is pushed to worst-case tolerance corners. Designs must use real manufacturer parts, with datasheet specifications carried into the SPICE models, so the agent has to satisfy electrical performance, cost and supply at once.

Deterministic grading

Every EEBench check is deterministic: the harness builds the submitted design, constructs the circuit graph and bill of materials, and runs SPICE simulations and design checks, with each requirement producing a measurement against a limit. The technical score is combined with cost efficiency against a reference bill of materials, and cost only counts once the circuit works. The team likens it to handing a coding agent a compiler and a test suite, except the tests measure voltages and component behavior. EEBench V1 covers analog and digital design through simulation only — layout, manufacturing and board bring-up are not yet tested, though the team plans to add them.

What the leaderboard shows

The September 1 results reported by EEBench put Claude Opus 5 first at 61.6% across the 13 tasks in EEBench V1, with Grok 4.6 second at 57.1%, just ahead of Claude Fable 5.1 at 56.4%. Claude Fable 5 scored 54.3% and Claude Opus 4.8 Max 51.4%. The OpenAI models tested so far sit further down the table, the post notes. Separately, xAI included EEBench in the Grok 4.6 model card under "engineering acceleration," alongside evaluations for 3D modeling and parametric CAD; its published run put Grok 4.6 at 60.0% using xhigh reasoning effort, a higher figure than EEBench's own September 1 result.

Why it matters

A frontier lab choosing a KiCad circuit board demo for its launch page signals that hardware design is becoming a showcase capability for AI, not just code. But demos are not measurements. EEBench turns "can it design electronics?" into a checkable question, and the honest answer today is partial: roughly 60% for the best model on a benchmark limited to design and verification through simulation. Two things make the approach credible — it forces the trade-off reasoning (tolerances, cost, ordering real parts) that defines actual engineering, and a frontier lab now cites it in a model card. The remaining gap is just as telling: layout, manufacturing and bring-up are untested, and, as EEBench itself puts it, asking an AI to build an entire phone in one prompt remains some distance away.