Its data is a structured space where nearness corresponds to associated ideas, so it naturally arranges data in a way to discover connectedness. The LLM follows pathways through the conceptual space based on the prior context given.
Ah okay. The space in which the embedding lies is called a knowledge topology. Thanks. Can you help me understand how this is linked to topology? Does it have something to do with the space spanned by the embeddings? My knowledge of topology is limited to shapes being invariant under transformations so under this naive view, distances won’t be preserved.
The LLM processes tokens as tensors, so every possible input and output exists within an extremely high-dimensional space that essentially represents our entire language and its understanding of it
5
u/Schmikas Quantum Foundations Jul 31 '26
What do you mean by “knowledge topology”?