Benchmark: AI agents build the same app for $2.67 in Hono+htmx, $4.97 in Angular

Nolan Lawson has published a benchmark that puts dollar figures on a question the web community tends to debate by feel: when an AI agent writes your app, does the framework you choose change what it costs? After having Claude build the same application repeatedly across twelve JavaScript stacks, his answer is yes — the priciest option came in at nearly twice the cheapest.

How the test was run

According to the post, published October 28, Lawson designed a compact but non-trivial Pokémon app on top of the PokéAPI: a searchable, filterable list of creatures showing types, moves and data from various games. He then instructed Claude Opus 5.5, run at high effort inside a sandbox, to build it with the latest version of a given framework or meta-framework — from plain JavaScript and a Hono-plus-htmx pairing through Next.js, Nuxt, SvelteKit, Angular and others.

To stop agents from gaming the exercise, every build was checked against a conformance suite based on the accessibility tree rather than the raw DOM, which also avoided prescribing HTML structure since one framework relies on shadow DOM. Where a framework ships an official MCP server or agent skills, Lawson supplied them, so that thin training-data coverage wouldn't be mistaken for framework quality. Agents were given the conformance tests but not told their output would be scored in Lighthouse.

Each framework was built five times and judged on the median; Angular is the exception because it failed to produce a working app in two runs, leaving three. The dollar figures are Claude Code's listed API prices rather than what Lawson actually paid — he was on a $100-per-month Max plan at the time.

What the agents cost

Median cost per build, with the range across runs:

Framework	Median	Range
Hono+htmx	$2.67	$2.23–$3.19
Astro	$2.76	$2.16–$3.63
Nuxt	$2.82	$2.39–$4.25
Vanilla JS	$2.98	$2.24–$3.37
Next.js	$3.14	$2.58–$4.13
Qwik	$3.27	$2.35–$3.80
TanStack Start	$3.54	$3.11–$3.96
SolidStart	$3.79	$2.84–$4.81
React Router	$3.81	$2.82–$3.97
SvelteKit	$4.28	$3.59–$4.65
Gracile+Lit	$4.65	$3.92–$5.07
Angular	$4.97	$3.19–$6.82

Lawson frames the full analysis around cost, quality and maintainability, and the cost numbers alone show heavy overlap: the cheapest React Router build ($2.82) cost less than the most expensive Hono+htmx build ($3.19), even though their medians sit about a dollar apart.

What drives the differences

Lawson attributes much of the noise to the randomness of language models — an agent occasionally goes down a rabbit hole and burns tokens for reasons unrelated to the framework — and to Lighthouse itself, which can pick different elements as the Largest Contentful Paint between runs.

Three stacks were consistently more expensive. SvelteKit agents read the v3 migration guide in every run, which Lawson suspects reflects a release too recent for the model's training data. Gracile agents barely knew the framework and resorted to cloning its repository or reading compiled source from node_modules. Angular's higher bill, he writes, tracks its larger total code size.

Agent tooling uptake was erratic. Nuxt's MCP server was never used; TanStack Start's agents found their skill files in only two of five runs; Next.js agents read a skill file once and an AGENTS.md twice. Lawson considers this realistic, since few developers badger their agents into using official tooling.

The post quickly reached the front page of Hacker News.

Why it matters

The absolute figures are tiny, but the pattern behind them is not. As agents write more production code, token consumption becomes a comparable, budgetable property of a framework — alongside bundle size and build time — and this is a rare attempt to measure it with conformance tests, repeated runs and honest error bars rather than anecdote.

The data also gives framework maintainers new leverage: fresh training data, a smooth upgrade path and a discoverable MCP or skill set translate directly into cheaper builds, while version churn or obscurity shows up as a tax. And Angular failing to ship a working app in two of five attempts is a reminder that reliability, not just price, varies by stack. Lawson is candid that one content-heavy, low-interactivity app cannot settle the SPA-versus-MPA argument — but it replaces a lot of assertion with numbers others can rerun.