deniz.in

Markets

Weather

Loading weather

· via TechCrunch

OpenAI chief scientist proposes perplexity as the yardstick for AI development limits

Jakub Pachocki argues that perplexity, a measure of how well a model predicts text, could serve as a shared benchmark that AI labs use to set enforceable limits on progress.

OpenAI chief scientist proposes perplexity as the yardstick for AI development limits

The proposal

Jakub Pachocki, OpenAI's chief scientist, has put forward a concrete mechanism for limiting AI development: adopt perplexity — a statistical measure of how well a language model predicts text — as an industry-wide yardstick, with labs committing not to push past a defined threshold until their ability to understand and monitor AI behavior reaches an agreed milestone.

According to TechCrunch, which reported on the new blog post, OpenAI CEO Sam Altman described it as the most important thing OpenAI has published in a while.

Perplexity, which has nothing to do with the search company of the same name, scores a model on its average accuracy at predicting the next token; lower values indicate better prediction. Pachocki's case for it starts from a complaint about how progress is currently tracked: as he argues, benchmarks that test what he calls "downstream capabilities" become outdated quickly as the technology advances, while perplexity has stayed informative.

His post includes a chart showing the perplexity of OpenAI's models falling since GPT-3.5, and he links the numbers to capability milestones. GPT-4, at roughly 0.91, was proficient enough at using chain-of-thought reasoning that OpenAI then deliberately elicited that ability, which led to the creation of modern reasoning models. Now, with losses below 0.8, he says capabilities he describes as transformative have emerged in fields such as cybersecurity and mathematics.

That history, in his framing, is what makes the metric a plausible common yardstick for general intelligence. If labs can agree on the number, they can agree on common limits — and on what must be achieved in understanding and monitoring models before anyone goes further.

The debate it enters

The post lands in the middle of an argument that has escalated sharply. According to TechCrunch, a former researcher who had worked at both OpenAI and Anthropic said the companies were "gambling with our lives," and Anthropic's own IPO filing acknowledged that the technology could pose catastrophic or existential risks to society.

Anthropic CEO Dario Amodei responded with a plan to "pace the frontier," urging labs to adopt common safety standards and "limits on the rate of unchecked AI progress." TechCrunch reports that the plan appeared to win endorsement from both Altman and Elon Musk.

What that plan lacked was a speedometer. A limit on the rate of progress requires agreement on what is being measured and how — which is precisely the gap Pachocki's post addresses.

What would have to happen

Several conditions would need to hold for the proposal to function in practice. The thresholds only bind if every frontier lab signs on; a limit observed by some companies and not others mostly changes who arrives first. Comparability is also a technical question rather than a given, since perplexity is measured against specific data, and the labs would need agreed evaluation sets for one company's number to mean the same thing as another's.

There is also a tension built into the pitch. Pachocki's own chart associates the arrival of transformative cybersecurity and mathematics capabilities with losses below 0.8 — a level OpenAI's models have already passed, by his own account. A commitment not to exceed a threshold only has teeth if the ceiling sits somewhere the frontier has not already been, and, as described by TechCrunch, the post frames the pause condition as a monitoring milestone rather than naming a specific cutoff.

Why it matters

Arguments about slowing AI development have largely stayed at the level of principle. This is a specific, measurable mechanism, proposed by the chief scientist of a lab at or near the frontier and flagged by that lab's CEO as significant. It also engages directly with a rival's framework: Amodei asked for limits on unchecked progress, and OpenAI is now answering with a candidate for the shared measurement those limits would need. Whether perplexity survives contact with competing labs — and whether anyone agrees to a ceiling they have not already blown through — will determine whether this becomes a real guardrail or a well-argued position paper.

  • #openai
  • #ai-safety
  • #language-models
  • #benchmarks
  • #anthropic

Related posts