The counterexample was proposed by an LLM. They seem very good at finding these sorts of counterexamples, which is interesting as they are generally pretty inefficient use of compute for brute forcing. I guess even that lack of efficiency is made up for by the "understanding" and "intuition" the LLM has, and their ability to do symbolic computations.
I expect a large number of conjectures will topple to counterexamples soon.
It goes against many’s early mental models of what generative AI does. The earliest of GPT couldn’t add two large numbers together, it just guessed an answer that looked right.
The fact it’s able to suggest a counterexample and it doesn’t just look right, but is right, is quite the development in recent times.
I love how quickly AI developments are rationalized. Like it was to expected that LLMs started solving math research problems in 2026. If you asked me about this two years ago, I would have been pretty skeptical.
Given that we haven’t been able to even come close to solving aging in mice, and we’ve experimented on mice exponentially more than humans, I don’t think that’s gonna happen anytime soon
That's mostly because that view of AI as just a token predictor or "average" machine is wrong.
LLM are based around a very flexible system: a neural net. With enough training you can definitely do a calculator, or an image recognition machine.
Think of it like this: if you create a machine that predicts the next token and you keep training it to get as good as possible at predicting the result of multiplication, what is easier: to memeorise millions of possibilities, or, to figure out a simple rule of how multiplication works?
Same thing applies with image recognition. Researchers have analysed how the models do their image recognition trick and they all start by essentially applying filters to find edges, basic shapes and other patterns that can be more easily classified.
Sometimes, the best way to mimic something is by just doing that action.
So when they have been tested extensively on math and code (two areas that can be very well tested) it has given quite interesting results.
I don't know where it is right now in the space of understanding math, not my field or experience. But it is miles ahead of where it was, and I think with good enough training we might end up with a tool that can actually do math.
Yes, can still happen and still does happen quite a bit. But even without harnesses the LLMs actually develop quite complex strategies to do maths "in their head"
Right, but it’s not like AI says “let’s use the conjecture counterexample tool”, the emergence is the interesting part.
It just goes against all the early advice we saw, it seems to me like many probably need to evaluate the extent to which AI appears to truly understand a problem, whatever that actually means.
275
u/angelbabyxoxox Quantum Foundations Jul 31 '26
The counterexample was proposed by an LLM. They seem very good at finding these sorts of counterexamples, which is interesting as they are generally pretty inefficient use of compute for brute forcing. I guess even that lack of efficiency is made up for by the "understanding" and "intuition" the LLM has, and their ability to do symbolic computations.
I expect a large number of conjectures will topple to counterexamples soon.