Mercor analysis finds 90% of US working hours go to tasks AI benchmarks don't cover

Roughly a tenth of US work time is actually benchmarked

About 90% of the hours Americans work are spent on tasks that current AI benchmarks measure weakly or not at all, according to research published by Mercor, which also maintains its own APEX evaluation suite. The study, which surfaced on Hacker News' front page, maps benchmark coverage across the US labor force and concludes that outside a handful of well-measured occupations, evidence about how capable AI models really are at everyday work is thin.

By Mercor's own metrics the gap is large: of the 923 occupations in the US Department of Labor's O*NET taxonomy, only 208 contain even one task with meaningful benchmark representation, and six of the taxonomy's 22 job families contain no such occupation at all.

How the measurement works

O*NET decomposes the US workforce into 923 occupations and 18,797 constituent tasks. Mercor assembled an index of 29 benchmarks selected for public availability, agentic task design, resemblance to real enterprise work, spread across job families, and difficulty that frontier models have not yet saturated. Per-family caps were applied, largely to offset the heavy overrepresentation of software engineering and research tasks in popular evaluations.

An LLM judge, following an approach from Wang et al., then tagged each benchmark prompt with the O*NET tasks it plausibly corresponds to; the automated tagging was validated against human expert labels on Mercor's APEX benchmarks. A task counts as covered only when at least five prompts map onto it. Coverage was finally combined with Hatgis-Kessell et al.'s estimates of how much working time each task consumes, producing the headline 90% figure.

Coverage tracks exposure and wage bills

What gets measured is systematic rather than random. Using AI-exposure labels from Eloundou et al. (2023), Mercor finds that tasks rated as exposed to language models are roughly six times more likely to appear on benchmarks than unexposed tasks. That helps explain why software engineering and financial analysis are dense with evaluations while truck driving and gas plant operation are nearly empty.

Exposure alone does not explain the pattern, though. Even after controlling for it in a task-level regression, tasks belonging to occupations with larger annual wage bills — computed from Bureau of Labor Statistics wage and employment figures — remain significantly more likely to be covered. Mercor's illustration: benchmarks map heavily onto lawyers' tasks but barely touch the tasks of legal secretaries or court clerks, jobs that carry similar exposure ratings. The upshot is a broad set of smaller, middle-income knowledge-work roles with almost no measurement at all.

Caveats the authors flag

Mercor describes the results as provisional. The estimates rest on an initial, incomplete list of 29 benchmarks, which pushes coverage estimates downward by an amount the researchers say they cannot bound. The per-family caps understate coverage in the most heavily benchmarked fields, such as software engineering. Excluding saturated benchmarks risks undercounting tasks that are genuinely easy for models, while including them could overstate coverage if saturation reflects low task realism; Mercor judges exclusion the lesser distortion but concedes neither option is clean. Selection criteria such as economic realism also involve subjective judgment.

Why it matters

Benchmark leaderboards anchor most public claims about AI capability, from lab marketing to automation forecasts. If the measured slice of the economy is this thin, and tilted toward large, high-exposure occupations, then headline scores say little about how models would perform in the jobs most people actually hold. Organizations weighing deployment should treat benchmark results as evidence about a narrow band of tasks rather than a general readiness signal. The research also hands benchmark builders a map of their blind spots, and Mercor says it plans to expand the index and revise the estimates as those gaps are filled.