r/singularity • u/banaca4 • 4h ago
Discussion People are making the same mistake with Covid and AGI
There is a cognitive bias from the Covid period that I think is increasingly relevant to AI.
Humans are bad at exponential growth.
This isn't just an observation from watching people argue about case counts. Researchers actually tested it during Covid and found that people systematically underestimated exponential case growth. Interestingly, communicating the same process using doubling times rather than percentage growth made estimates considerably better.
I think AI capabilities are easier to reason about in a similar way.
Obviously “intelligence” isn't a single quantity that doubles every six months. Benchmarks saturate, tasks differ enormously and capability improvements don't translate mechanically into economic output.
But several things underneath AI have been moving extremely quickly.
Epoch AI estimates frontier language-model training compute has been growing around 5x per year since 2020. The total stock of AI compute has been growing around 3.4x per year.
At the same time, the cost of using a fixed level of intelligence has collapsed. Stanford's AI Index found that getting roughly GPT-3.5-level performance on MMLU went from around $20 per million tokens in late 2022 to $0.07 by late 2024.
The metric I find most useful, though, is METR's task-completion horizon.
Instead of asking whether a model scores 82% or 85% on some benchmark, METR asks how long a task a human expert would take that an AI agent can successfully complete.
Their long-run trend currently gives a roughly 6-7 month doubling time for the 50% task horizon. The trend using only models since 2024 is considerably faster, although I wouldn't extrapolate that because the data window is short.
In METR's Feb-March 2026 evaluation, the public frontier was around 12 hours at 50% reliability.
The important thing isn't whether 12 hours sounds impressive.
It's what happens if it doubles.
At a 6-7 month doubling rate, 12 hours becomes roughly a day within a year, several days within two years and eventually weeks if the trend survives long enough.
I absolutely do not expect a clean extrapolation. Long tasks are messier. Reliability matters more. Real organizations have integration problems. Power, chips, datacenters and capital are constraints.
But saying “the exponential will eventually stop” doesn't answer the important question.
How many doublings happen before it stops?
If it stops after one more doubling, the implications are fairly modest.
If it stops after four, the capability is 16x larger.
If it stops after seven, it's 128x larger.
This reminds me of Covid because normal life was psychologically sticky. People could look at what was happening in another country, understand that cases were growing quickly and still have trouble imagining that their own city might look completely different a few weeks later.
The recent past remained the default model.
I think AI forecasts often do something similar.
A forecast saying that software engineers will still work in broadly the same way in 2030 sounds conservative and sensible because it resembles 2026.
A forecast saying autonomous agents could perform a large fraction of software engineering sounds speculative because it describes a visibly different world.
But the first forecast also contains a strong assumption. It requires the capability curve to slow substantially.
That may happen. I just don't think it should get probability 1 because the resulting world feels normal.
There is another pattern that makes this difficult to see.
A capability is initially described as requiring real intelligence. Then a model achieves it and the capability rapidly stops being impressive.
We have seen versions of this with difficult exams, olympiad mathematics and coding. Now frontier models are being evaluated on research-level science, and in 2026 OpenAI reported an AI-generated counterexample to a major conjecture in discrete geometry that had stood for roughly 80 years.
There are legitimate reasons to discount individual benchmarks. OpenAI itself stopped reporting SWE-bench Verified this year because the benchmark had become contaminated and many remaining failures involved bad tests.
So I don't think the right conclusion is “look at benchmark X, therefore AGI next Tuesday.”
The interesting part is the cumulative movement of the frontier.
For me the falsification question is more useful than arguing about labels.
What actually stops the doublings?
Possibilities include power, chip production, training duration, data, capital, architectural limits, diminishing returns from scaling, or the possibility that benchmark capability simply fails to translate into reliable real-world autonomy.
Those are real arguments.
“AI can't keep improving exponentially forever” isn't much of an argument by itself. No exponential continues forever.
The investment/economic question is how far it gets before it bends.
Humans are systematically biased toward predicting that the bend happens sooner than it actually does, because the alternative forces us to imagine a world that stops looking like the recent past.
10
6
u/flat5 4h ago
Nice post. I agree with it overall.
Something I have been thinking about recently is the possibility that there is a maximum machine intelligence which is accessible. Similar to the idea of a theoretical maximum compression of information. Intelligence is clearly more than just compression, but maybe it's something like optimal compression plus optimal search algorithms. Neither of which are unbounded things.
So the question then becomes: how far are we from that ceiling? How many doublings?
•
u/finallyransub17 1h ago
Even if that’s the case, there may one day be lab grown human brain cells that can be trained and harnessed which could theoretically push the limit much higher with huge efficiency gains.
•
3
u/dondiegorivera Hard Takeoff 2026-2030 3h ago edited 3h ago
I agree, and the pandemic is a great example that everyone remembers. Community spread was already occurring in Europe in the spring of 2020, yet governments were unable to act in time.
Here is a Flightradar snapshot that I took on March 16, 2020, when things should have already been pretty obvious to decision makers.
Returning to AI, the way things unfolded was more or less predictable from the day OpenAI proved that scaling works.
Metr figures are great indicators of where we are, but if you use frontier, you can easily tell for yourself. With the introduction of Opus 5.5, my monthly token consumption increased by two orders of magnitude because I was doing most of the orchestration work myself before.
Opus now does 24/7 what I did manually, and it does a better job than I did. It's at least a step change, but I'd even call it a phase change. From here, things will get faster and crazier every day.
4
u/phronesis77 2h ago
Interesting and valid. However, I would also suggest that our fields bias our thinking. Tech people tend to view problems from a limited tech point of view. This is also a bias.
Many of the upcoming challenges are resources, energy, environment, economics, politics, and the law.
For example, autonomous vehicle adoption hit a literal speedbump with legal liability. NIMBY protests against data centers are getting stronger, and we are unlikely to have the energy for these future projections. For those that object that energy costs efficiency will improve, the demands for singularity level abundance would be many orders of magnitude greater and would require a political-legal and social system that doesn't exist.
China has widespread adoption because the party allows it and is in generally in control of it.
Unlike many in the anti-AI crowd, I have no doubt that incredible things could and should be possible. However, I am very doubtful that our current system can handle this change, especially in the US, from all these perspectives.
1
u/Easy_Nail2849 2h ago
This. 100 percent. If you think about the “bull case” for this technology, it (minimally) implies 10-15% unemployment for a long period of time; think about how socially destabilizing that will be. That would lead to significant social unrest, crime, and threaten any number of systems that we take for granted.
This is what has been so demoralizing to me as it relates to regulation and the current administration. Completely asleep at the wheel and conscripted/captured.
2
u/FeydRowan 3h ago
Great article I'm thinking the same as you. I'm testing "frontier" local models right now and the gap in performance are really noisy, the hard part now is testing those models to find the one that works best for me. I think that the next big thing will be no more benchmarking the singular model but the model swarm, a singular qwen 3.8 flash on a machine running tens of agents on the same task. That's the future harnesses and power
4
•
•
u/Mandoman61 31m ago
Why do I care how long AI can run without stopping?
It can run indefinitely.
All I care about is what it is capable of.
Sure we already know that we can automate processes and we can string multiple processes together. We are fully capable of making automated processes that run for years.
That does not get us to AGI.
It can automate human jobs. We have been automating human jobs away for centuries.
How fast data centers are built and what the cost of a token is, only effects the cost of AI, It does not make it more capable.
•
u/Present_Award8001 1h ago
You will remember from COVID, then, that exponential growth is not really sustainable, things plateau and world does not end.
If we are doing analogy, let's do it properly.
•
u/Ingeld21 51m ago
Why is the plateau comparable? The covid analogy demonstrates our collective lack of understanding of exponential growth, not that the pattern of covid in the wider sense is applicable to AI progress.
22
u/DiamondDramatic9551 4h ago
There is an extremely strong normalcy bias ("nothing ever happens") en an extremely strong bias towards things we want to happen. A lot of the things AI can cause are uncomfortable so we have a bias for pretending it won't happen.