No data commons exists for posttraining, so agentic RL data must be manufactured

A new analysis from OpenAthena, which drew front-page attention on Hacker News, makes a blunt argument about where AI research stands: the field got its start by consuming a public data commons that already existed, but the work now dominating frontier labs — posttraining, especially for agentic systems — has no equivalent trove to draw on.

The pretraining windfall

The essay opens by crediting luck. Large language models required enormous text corpora, and decades of digitization had already produced one: Wikipedia's curated encyclopedia, petabytes of open-source code, more than a decade of question-and-answer curation on Stack Exchange, scientific preprints on arXiv, and Project Gutenberg's digital books — all sitting atop Common Crawl, which has archived regular web snapshots since 2007 and now holds roughly 10 petabytes.

According to the essay, GPT-3's training data was about 60 percent Common Crawl-derived, and the original Llama mixture came almost entirely from the internet commons. Although large incumbents now operate custom crawlers — Cloudflare reports that over half of crawler traffic is AI-related — modern open models such as Nemotron, and OpenAthena's own, still rely on that commons for the vast majority of their pretraining data. The authors' conclusion: without continued access to it, truly open LLMs cannot exist.

Documents versus tasks

The structural difference, per the analysis, is that pretraining consumes documents while posttraining consumes tasks — and tasks do not naturally persist. Human tasks are situated: someone wants something, acts on the world, adapts as circumstances shift, and the moment never recurs in quite the same form. Encoding an activity as a reinforcement-learning task freezes it, the essay argues, much as transcribing an oral story preserves one telling while losing the living performance.

Every RL task requires a preserved world, an intent, and a set of tools, plus a grader that scores how well the intent was satisfied. Humanity has spent millennia building institutions around durable documents, and far less time building infrastructure for durable action.

Software as the near-exception

One field has spent decades writing tasks down: software engineering. Version control preserves history, continuous integration and unit tests record procedures, and containers encapsulate controlled environments — all resting on the determinism that lets an agent retry a task from an identical starting point, which is exactly what RL algorithms favor. Even so, the essay notes software is imperfectly durable: dependencies vanish, bits rot, and Debian has worked on reproducible builds since 2013 without describing the distribution as fully reproducible.

The SWE-bench funnel

In theory, open-source repositories supply every ingredient: the repository at a point in time is the world, the shell and compiler are the tools, issues and pull requests recover the intent, and passing tests serve as the grader — essentially the original SWE-bench framing. The figures the essay cites show how lossy that conversion is. SWE-bench's authors collected roughly 90,000 pull requests from 12 Python repositories, and only about one in eight could even be mapped to the frame; many were unmerged, unlinked to issues, or changed no tests. Of the 11,407 that remained, around 80 percent could not be turned into executable tasks because environments could not be reconstructed or graders failed, leaving 2,294 tasks.

A year later, OpenAI hired 93 experienced Python developers to manually review 1,699 randomly sampled tasks. More than two-thirds were flagged for removal over underspecified prompts, unfair tests, or other major issues, producing 500 "Verified" tasks — and a later audit of a problematic subset found flaws in roughly another sixth of the tasks.

The lesson the essay draws is that the missing information often was never recorded at all, which makes cleaning web text and building tasks qualitatively different problems. Posttraining data work looks less like curation and more like manufacturing.

Why it matters

If capability gains — and much of what gets called alignment — now come from posttraining, the scarce input is not scraped text but manufactured, gradable tasks. That reshapes who can compete: organizations able to fund environments, graders, and human review at scale hold an advantage that public data cannot offset, and open efforts that depended on the pretraining commons have no equivalent resource to mine for posttraining. The practical question going forward is not just how much compute posttraining consumes, but who can build and verify the task data it feeds on — and whether anything like a genuine commons of durable tasks can be constructed at all.