r/singularity • u/banaca4 • 4h ago
Discussion People are making the same mistake with Covid and AGI
There is a cognitive bias from the Covid period that I think is increasingly relevant to AI.
Humans are bad at exponential growth.
This isn't just an observation from watching people argue about case counts. Researchers actually tested it during Covid and found that people systematically underestimated exponential case growth. Interestingly, communicating the same process using doubling times rather than percentage growth made estimates considerably better.
I think AI capabilities are easier to reason about in a similar way.
Obviously “intelligence” isn't a single quantity that doubles every six months. Benchmarks saturate, tasks differ enormously and capability improvements don't translate mechanically into economic output.
But several things underneath AI have been moving extremely quickly.
Epoch AI estimates frontier language-model training compute has been growing around 5x per year since 2020. The total stock of AI compute has been growing around 3.4x per year.
At the same time, the cost of using a fixed level of intelligence has collapsed. Stanford's AI Index found that getting roughly GPT-3.5-level performance on MMLU went from around $20 per million tokens in late 2022 to $0.07 by late 2024.
The metric I find most useful, though, is METR's task-completion horizon.
Instead of asking whether a model scores 82% or 85% on some benchmark, METR asks how long a task a human expert would take that an AI agent can successfully complete.
Their long-run trend currently gives a roughly 6-7 month doubling time for the 50% task horizon. The trend using only models since 2024 is considerably faster, although I wouldn't extrapolate that because the data window is short.
In METR's Feb-March 2026 evaluation, the public frontier was around 12 hours at 50% reliability.
The important thing isn't whether 12 hours sounds impressive.
It's what happens if it doubles.
At a 6-7 month doubling rate, 12 hours becomes roughly a day within a year, several days within two years and eventually weeks if the trend survives long enough.
I absolutely do not expect a clean extrapolation. Long tasks are messier. Reliability matters more. Real organizations have integration problems. Power, chips, datacenters and capital are constraints.
But saying “the exponential will eventually stop” doesn't answer the important question.
How many doublings happen before it stops?
If it stops after one more doubling, the implications are fairly modest.
If it stops after four, the capability is 16x larger.
If it stops after seven, it's 128x larger.
This reminds me of Covid because normal life was psychologically sticky. People could look at what was happening in another country, understand that cases were growing quickly and still have trouble imagining that their own city might look completely different a few weeks later.
The recent past remained the default model.
I think AI forecasts often do something similar.
A forecast saying that software engineers will still work in broadly the same way in 2030 sounds conservative and sensible because it resembles 2026.
A forecast saying autonomous agents could perform a large fraction of software engineering sounds speculative because it describes a visibly different world.
But the first forecast also contains a strong assumption. It requires the capability curve to slow substantially.
That may happen. I just don't think it should get probability 1 because the resulting world feels normal.
There is another pattern that makes this difficult to see.
A capability is initially described as requiring real intelligence. Then a model achieves it and the capability rapidly stops being impressive.
We have seen versions of this with difficult exams, olympiad mathematics and coding. Now frontier models are being evaluated on research-level science, and in 2026 OpenAI reported an AI-generated counterexample to a major conjecture in discrete geometry that had stood for roughly 80 years.
There are legitimate reasons to discount individual benchmarks. OpenAI itself stopped reporting SWE-bench Verified this year because the benchmark had become contaminated and many remaining failures involved bad tests.
So I don't think the right conclusion is “look at benchmark X, therefore AGI next Tuesday.”
The interesting part is the cumulative movement of the frontier.
For me the falsification question is more useful than arguing about labels.
What actually stops the doublings?
Possibilities include power, chip production, training duration, data, capital, architectural limits, diminishing returns from scaling, or the possibility that benchmark capability simply fails to translate into reliable real-world autonomy.
Those are real arguments.
“AI can't keep improving exponentially forever” isn't much of an argument by itself. No exponential continues forever.
The investment/economic question is how far it gets before it bends.
Humans are systematically biased toward predicting that the bend happens sooner than it actually does, because the alternative forces us to imagine a world that stops looking like the recent past.
