Why AI Benchmarks Miss 90% of Real Work

A friend of mine runs operations at a mid-sized logistics company, and last spring her team spent three weeks evaluating whether an LLM could take over their exception-handling workflow — the unglamorous work of reconciling invoices that don't match purchase orders across three systems that hate each other. The vendor's deck was full of impressive numbers: MMLU, GSM8K, HumanEval, a smattering of agentic benchmarks. The model aced every canned test they threw at it. Then they pointed it at a real week of exceptions and it confidently matched a $14,000 invoice to the wrong PO because a vendor had abbreviated their own name differently. Nobody's leaderboard measures that.

That gap is now measurable. A recent mapping of public AI benchmarks against the US Department of Labor's O*NET taxonomy — all 923 occupations and 18,797 tasks that make up American work — found that 90% of US working hours are spent on tasks with little or no benchmark coverage. Only 208 of 923 occupations have even a single task backed by five or more benchmark prompts. Six of the 22 job families have none at all. The uncomfortable conclusion isn't that AI is weak. It's that our instruments for measuring AI capability are pointed almost entirely at the wrong part of the economy.

What AI Benchmarks Actually Cover

The methodology behind this mapping is worth understanding because it's more careful than the headline suggests. Researchers took 29 high-quality, unsaturated, agentic benchmarks — tests where models must actually perform work rather than answer questions — and used an LLM judge (validated against human experts) to tag each benchmark prompt with the O*NET tasks it exercises. A task counts as "covered" if at least five prompts map to it. Combine that with estimates of how much working time each task consumes, and you get an economy-wide picture of what our evaluation suite can see.

What it can see is narrow and tall, not broad. Software engineering and financial analysis are saturated with benchmarks. Truck driving, gas plant operation, food service, and most of healthcare's daily work are essentially invisible. And the skew isn't just about which tasks are technically automatable. Tasks tagged as "AI-exposed" are about six times more likely to be benchmarked than unexposed ones — which makes sense — but even after controlling for that, tasks from occupations with large total wage bills are significantly overrepresented. Lawyers' tasks are heavily covered. Legal secretaries and court clerks, who face similar AI exposure, barely appear.

Read that again: we're benchmarking AI against the prestigious version of a job while ignoring the version most people actually hold. The long tail of mid-income knowledge work — the coordinators, clerks, specialists, and schedulers whose hours add up to an enormous share of the economy — has almost no measurement infrastructure at all. When someone claims AI can do some percentage of "knowledge work," ask which knowledge work, measured how, against what task list. The honest answer is usually: a thin, unrepresentative slice, measured against tests designed for research papers.

Benchmark Saturation Is an Evaluation Design Failure

I spent years building evaluation pipelines, and the hardest lesson was this: a benchmark is a model of the world, and most benchmarks are terrible models. They compress a messy, contextual, multi-step human task into a clean prompt with a checkable answer, because that's what's cheap to grade. Then Goodhart's law does the rest. The moment a benchmark becomes the target for model development, it stops measuring the capability it was designed to proxy. Training data absorbs the test distribution, teams tune against the leaderboard, and scores climb while real-world performance jogs sideways.

Saturation is the visible symptom. When every frontier model scores above 90% on a test, the test has stopped discriminating — but the deeper problem is that we respond by building another benchmark in the same style, covering the same kinds of tasks, because those are the tasks we know how to grade automatically. Multiple-choice QA begets harder multiple-choice QA. Coding challenges beget harder coding challenges. Meanwhile, nobody builds the benchmark for "notice that this contract's indemnification clause quietly contradicts the term sheet from March," because grading that requires a lawyer, time, and judgment calls that resist automation.

This is why I disagree with the framing that benchmark saturation means AI has "solved" a domain. Saturation means the instrument is maxed out, full stop. A thermometer that pegs at 40°C hasn't proven the room is 40 degrees; it's proven you need a different thermometer. If you're evaluating models for production use — and especially if you're building LLM-based decision systems where errors have real costs — leaderboard deltas on saturated benchmarks should carry almost zero weight in your decision.

We know precisely how good models are at the work we've decided to measure. We know almost nothing about the other ninety percent.

The Streetlight Effect in AI Evaluation

There's an old joke about a drunk searching for his keys under a streetlight — not because he lost them there, but because that's where the light is. Benchmark construction follows the same logic, and the data confirms it isn't random. The mapping study found that task importance within an occupation predicts coverage: O*NET tasks rated most important to their occupation are six to ten times more likely to be benchmarked than the least important ones. On its face, that sounds like sensible prioritization. In practice it means we measure the showcase tasks and skip the connective tissue.

Every job is mostly connective tissue. A software engineer's benchmarked task is "implement a function that passes these tests." Their actual day includes reading a colleague's half-finished migration, figuring out which stakeholder actually owns a decision, and noticing that the ticket describes the wrong problem. I've written before about how agentic coding tools succeed or fail on exactly this: the loop, the context, and the permission boundaries matter more than raw generation quality. None of that shows up in a HumanEval-style score.

There's also a subtle definitional problem in the coverage numbers themselves. "Covered" means five or more prompts map to a task — not that five prompts capture the task. Thirteen prompts about counseling parents on child-rearing problems do not represent what a social worker actually does across a caseload, a courtroom, and a kitchen-table visit. If you applied a layperson's definition of "covered well," the measured share of the economy wouldn't be 10%. It would round to zero. The streetlight isn't just narrow; the light it casts is thin.

Models Generate, but Nobody Benchmarks Judgment

Here's the asymmetry I find most damning. Nearly every benchmark in existence tests production: write the code, draft the memo, solve the problem. But the economically critical half of most real tasks is detection — catching the mistake, flagging the inconsistency, knowing when the confident answer is wrong. A junior accountant's value isn't producing a reconciliation; it's noticing that a number smells off. Benchmarks almost never test that, because building a graded error-detection task is much harder than building a graded generation task. You need realistic flawed artifacts, plausible-but-wrong subtlety, and human agreement on what counts as caught.

This is the doorman fallacy applied to evaluation design. Replace a doorman with an automatic door and you discover, too late, that the doorman was also greeting tenants, accepting packages, and watching the street. Automate the benchmarked part of a job and you discover the unbenchmarked parts were load-bearing. My logistics friend's model could draft excellent reconciliation notes — the visible task — while silently botching the matching, the judgment that made the notes worth writing. The eval suite scored the garnish and ignored the meal.

Until error detection gets first-class benchmark treatment, expect a persistent gap between demo performance and deployment performance. It's the same gap that lives between what a specification says and what the implementation does — and just like in software, the gap is where the failures breed. A model that generates at the 90th percentile and verifies at the 30th is, in production, a 30th-percentile employee with a typing speed advantage.

Weighting Evals by the Real Task Distribution

The most practical idea in the benchmark-mapping work is also the simplest: if benchmarks overrepresent some tasks, stop averaging over prompts and start averaging over tasks, weighted by how much working time each task actually consumes. A naive average of every prompt tagged "financial analyst" is dominated by whichever task happens to have the most benchmark prompts — usually the one that was easiest to build a dataset for. An O*NET-weighted average gives each real task equal footing. The difference can be dramatic.

import pandas as pd
# The tasks an occupation actually performs, with time shares
# (from O*NET, your own ticket logs, or time-tracking data)
tasks = pd.DataFrame({
"task": ["draft contract", "advise client", "review filing", "coordinate closing"],
"time_share": [0.30, 0.25, 0.35, 0.10],
})
# Model results: one row per eval prompt, mapped to the task it exercises
results = pd.DataFrame({
"prompt_id": ["p1", "p2", "p3", "p4", "p5"],
"task": ["draft contract", "draft contract", "draft contract",
"review filing", "coordinate closing"],
"score": [0.92, 0.88, 0.95, 0.61, 0.40],
})
# Naive: flat average over prompts. Drafting has 3 of 5 prompts,
# so the score is dominated by the model's best task.
naive = results["score"].mean()  # 0.752
# Task-weighted: average within each task first, then weight by
# the share of working hours that task consumes.
per_task = results.groupby("task")["score"].mean()
weights = tasks.set_index("task")["time_share"]
weighted = (per_task * weights).sum() / weights[per_task.index].sum()
# 0.30*0.917 + 0.35*0.61 + 0.10*0.40, over covered time (0.75) = 0.705
# vs naive 0.752 — and 'advise client' (25% of the job) has no data at all

Two things to notice in that snippet. First, the weighted score drops because the model's weakest tasks were underrepresented in the prompt pool — exactly the distortion a flat average hides. Second, and more important: "advise client" vanishes from the calculation entirely because no prompt maps to it. Any weighting scheme silently drops tasks with zero coverage, and zero coverage is the norm for most of the economy. Weighting fixes the weighting problem. It cannot fix the missing-data problem. That requires building new evals, not re-averaging old ones.

How to Evaluate AI Against the Work You Actually Do

If you're choosing or deploying models, the economy-wide coverage gap has a concrete implication: public benchmarks are a weak prior for your use case unless you happen to be in the small, bright corner they measure. So build your own corner. This is less heroic than it sounds, and it beats trusting a vendor deck by a wide margin.

• Sample tasks from logs, not imagination. Pull a week of real tickets, emails, or exceptions and sample by frequency. Your eval set should look like your workload's histogram, including the boring tail — the boring tail is most of the job.
• Weight by time or cost, then report both numbers. A flat average and a weighted average bracket the truth. If a model wins the flat average but loses the weighted one, it's good at the tasks that are easy to test, which is precisely the failure mode you're screening for.
• Include verification tasks. Take real outputs — including known-bad ones — and score the model on catching planted and natural errors. Detection skill and generation skill are barely correlated in my experience, and detection is what you're actually buying.
• Hold out a private set and refresh it. Any eval you reuse for months becomes a training target, even unintentionally, the moment someone starts tuning prompts against it. Rotate cases quarterly.
• Score end-to-end outcomes where possible. If the task is "reconcile this invoice," grade the reconciliation, not the intermediate prose. Intermediate metrics flatter models the way unit tests flatter code with no integration coverage.

There's a final, quieter effect worth naming, one that anyone using AI coding tools daily will recognize: micro-scale Baumol cost disease. When generation gets twenty times faster, you don't spend your day generating. You spend it on the parts that didn't speed up — reviewing, coordinating, verifying, deciding. The unbenchmarked remainder doesn't shrink as AI improves; it grows to fill the freed time, and it becomes a larger share of what determines whether the AI was worth deploying. Measuring the automatable slice with ever-greater precision while ignoring the remainder isn't just incomplete. It inverts what matters.

None of this is an argument that AI progress is fake — the capability gains are real, and I've staked projects on them. It's an argument that our measurement culture is pointed at the wrong distribution. Benchmarks got built where prompts were cheap to write and grade: code, math, exam-style knowledge. The economy is mostly not that. Until evaluation catches up, treat every headline capability claim as a statement about the bright tenth of the map, and build the evals that cover your own dark ninety. The organizations that do this boring, unglamorous measurement work will make better deployment decisions than everyone refreshing leaderboards — and it won't be close.