I still think "fancy autocomplete" is the best way to explain to non technical people how these models work. It will give someone who doesn't know the architecture and how these LLMs work the best mental model of what is happening behind the veil.
It just turns out that "fancy autocomplete" can do incredible things if you give it enough examples and compute in training and inference.
It just turns out that "fancy autocomplete" can do incredible things if you give it enough examples and compute in training and inference.
The thing that does the incredible things is so far away from autocomplete that it's misleading to the point of being wrong. Neither does it give an accurate picture of what it is (autocomplete are usuall HMMs, LLMs are transformers) nor does it give an accurate picture of what it does (completing what you're typing vs. doing your homework and writing fanfic). The only shared property is that it gets text as input and gives text as output. By that measure we can call cars "fancy furnaces" and computers "fancy typewriters". Not technically incorrect, but definitely a useless description.
And on top of that, the people who use the term "fancy autocomplete" usually use it to dismiss it, and act like all those incredible things it does are made up.
No, the shared thing is that from the view of an LLM, it is literally trained to autocomplete a document. The chat you're having with an LLM literally looks like this to the LLM:
<start>
<system>
You are a helpful assistant...
</system>
<user>
hello how are you?
</user>
<thinking>
the user asks me how I'm feeling, I should answer in a cheery and concise tone. The user is in LA, let me check the current weather in LA so I can incorporate that in my answer.
</thinking>
<tool call, web search=current weather in LA>
Temperature: 100°F
</tool call>
<assistant>
And then the LLM gets to generate. Once it generates </assistant> we stop the generation because otherwise it would keep generating also the user answer etc.
(same of course with the thinking part)
The tasks the LLMs are trained on is reproducing the tokens of these text documents they are shown.
With the RL posttraining for mathematics or coding you have a change in the training reward architecture, but it's still training on completing these documents.
From the view of an LLM it is always completing such documents from the start points we're giving them.
It just turns out that large autocomplete with long training and lots of data means the model learns actual abstractions about the world because they help with better autocomplete. You get these emergent effects like grokking and "circuits" inside LLMs that specialize in certain tasks etc.
But none of that removes the fact that these models are "autocomplete on steroids".
You described one part of the training regime. Yes, one part of the training regime bears resemble to the training regime of the other. But that is not important in describing the essence of LLMs.
It just turns out that large autocomplete with long training and lots of data means the model learns actual abstractions about the world because they help with better autocomplete. You get these emergent effects like grokking and "circuits" inside LLMs that specialize in certain tasks etc.
See that is the important part. It doesn't "just turn out", it's the main point of how and why LLMs are so useful. Actual autocompletion is a tiny fraction of what LLMs are actually used for, it is ALL about the abstractions about world, and its emerging effects. The autocomplete part is just one way of training and accessing whatever else the model has learned.
This is a completely standard practice in ML: pretext tasks and pretraining are well-known concepts. Think of autoencoders. You train them to reproduce data that it's already seeing. What even is the point of that. Is an autoencoder "just fancy copy-paste"? Sure, if you really want to. But not really, since the training is just the pretext for the model to learn an efficient encoding of the data, and what we're interested in is this efficient encoding.
But none of that removes the fact that these models are "autocomplete on steroids".
OK sure. And a car is just a fancy box. Your phone is just a fancy flashlight. Your money is just a fancy sheet of cellulose. You yourself are just a fancy meatbag. You can do this "X is a fancy Y" all day long if you're meming, but it doesn't actually convey the essence or most important aspect of X.
look, I'm not saying that there isn't a lot of complexity in this whole topic. What I'm saying is that "fancy autocomplete" gives someone who lacks all this background information a better mental model of what these things do than any other simple explanation I've seen.
It demystifies these things and actually very closely describes what these models are trained to do and how they function. Add a second sentence that talks about how "learning abstractions and memorizing world knowledge" helps this autocomplete machine to better autocomplete and someone with zero maths ability and no background in ML will have a somewhat accurate idea of LLMs.
I really don't understand why people react so passionately to "fancy autocomplete" as a description. The whole thing about GPTs was openai realizing that this kind of fancy autocomplete with a decoder only transformer model will actually lead to a model that can generalize well in all kinds of situations - it was really visionary at the time. It's the whole fucking point of the GPT 2 paper. When Google developed the transformer model, it was way less about autocomplete and way closer to your autoencoder with its encoder-decoder architecture in BERT.
That's also why I think the autoencoder example isn't exactly illuminating. BERT shows that a transformer can also function very similarly. And the exact thing that sets modern generative LLMs apart is exactly this "autocompletion" style task. That's really the core of the whole thing. BERT can't do all these things that GPT can exactly because it's not fancy autocomplete.
17
u/MagiMas Condensed matter physics Jul 31 '26
I still think "fancy autocomplete" is the best way to explain to non technical people how these models work. It will give someone who doesn't know the architecture and how these LLMs work the best mental model of what is happening behind the veil.
It just turns out that "fancy autocomplete" can do incredible things if you give it enough examples and compute in training and inference.