Why do AI models make things up?
Because a language model is trained to produce plausible continuations of text, not to state truths — and fluent plausible text and accurate text are different targets that usually coincide and sometimes do not.
The mechanism. The model learns statistical relationships between tokens across an enormous body of text. Given some input, it produces what is likely to come next. It has no separate store of facts it consults and no mechanism that distinguishes knowing from not knowing — both produce the same kind of output.
Why fabrications sound confident. Confident text is what confident text looks like in the training data. A wrong answer is generated by exactly the same process as a right one, so there is no internal signal of uncertainty expressed by default. This is the property that makes the failures dangerous: the wrongness is not visible in the output.
Where it happens most:
Specific verifiable details — citations, statistics, dates, quotations, case names, URLs. These follow strong patterns, so a plausible-looking one is easy to generate and the pattern is all the model needs.
Rare or absent information. Where training data is thin, the model interpolates.
Questions with a false premise, which the model tends to accept and build on rather than challenge.
Long outputs, where consistency degrades.
Why the term "hallucination" is contested. It implies a malfunction, when the behaviour is the system working as designed — the model is always generating plausible text, and "correct" is not a category it operates on. Some researchers prefer confabulation or simply fabrication.
What reduces it:
Retrieval-augmented generation, supplying source documents so the model summarises given material rather than generating from parameters. This is the most effective single technique, though it can still misrepresent what it was given.
Asking for sources and checking them, since fabricated citations are the most detectable failure.
Constraining scope, and permitting "I don't know".
Lower temperature settings.
What does not work: asking the model whether it is sure. A model's confidence statement is generated by the same process and is not an introspective report.