1. Not "Talking Nonsense," but "Taking a Probabilistic Gamble"
Under the Transformer architecture, the model doesn't "store" facts like a database — it stores probability distributions.
When we input "Qin Shi Huang not only unified the six kingdoms, but also..." the model engages in Next Token Prediction. It searches the immense high-dimensional space for the most plausible next word.
- Fact: This typically corresponds to the highest peak in the probability distribution — for example, "built the Great Wall."
- Hallucination: This occurs when the model selects a slightly less probable but still logically coherent path — for instance, "invented the light bulb."
Mathematically, the probability of generating a sequence $Y = (y_1, y_2, ..., y_n)$ is:
The so-called "hallucination" is essentially the model deviating from maximum likelihood estimation during the sampling process in pursuit of diversity. If the model always chose the single most probable token, its responses would be painfully dull, repetitive, and rigid.
2. Temperature: The Dial Between "Precision" and "Madness"
This brings us to a critical parameter: Temperature.
- Low temperature ($T \to 0$): The model becomes conservative, sticking to combinations it saw most often in training data. It's a meticulous scribe — few hallucinations, but zero creativity.
- High temperature ($T > 1$): The model is allowed to explore the "long tail" of the probability distribution. It starts connecting unrelated concepts. It becomes a mad poet.
Core insight: Creativity, at its essence, is "unexpected yet plausible." Human inspiration often arises from a "misconnection" or "associative leap" between neurons. Therefore, hallucination is not a system failure — it's a byproduct of the system attempting to be creative.
3. Compression and Lossy Reconstruction: The Fuzzy Zone of Memory
Ted Chiang once compared ChatGPT to a "blurry JPEG of all text on the web." Transformer models compress vast amounts of human knowledge into weight matrices. This compression is lossy.
When the model tries to reconstruct a specific fact (say, the third author of an obscure paper), it has lost the exact "pixels" due to compression and retained only the outline. So it uses its powerful generalization ability to "fill in" the missing pixels.
This "filling in" ability is called "imagination" when writing science fiction, and "fatal hallucination" when writing medical reports.
4. If We Can't Eliminate It, Let's Harness It
If we accept that hallucination is the twin sibling of creativity, then the engineering goal shouldn't be to "eliminate hallucination entirely" (which would mean killing the model's generalization ability), but rather to "control the context in which hallucination occurs."
- RAG (Retrieval-Augmented Generation): Give the model a "reference book," forcing it to consult external material before answering factual questions. This uses an external knowledge base to constrain internal probability divergence.
- Chain of Thought: Have the model show its reasoning steps, forcing the probability distribution to converge toward logically self-consistent paths.
5. Conclusion: The Machine's "Dream"
When humans dream, the brain cuts off sensory input, and internal neurons fire freely, producing surreal hallucinations. Yet many great scientific discoveries and artistic inspirations have come from dreams or daydreams.
AI hallucination is the machine's "dream." When we accuse AI of "hallucinating," we're really accusing it of exhibiting idealist (creative) tendencies in a context that demands materialism (factual accuracy).
Understanding this is key to truly understanding Transformer: it's not a search engine that merely looks things up — it's a probability-based generation engine. Its nature is to generate; accuracy is merely a constraint that humans impose on it.
← Back to Articles