Lesson 13 of 28 · Language models and chat assistants
What is an AI hallucination?
An AI hallucination is when an AI model produces information that sounds confident and plausible but is false, made up or not supported by its sources, such as an invented quote, a wrong date or a citation to a paper that does not exist. It happens because language models generate likely-sounding text rather than looking facts up and checking them.
Why AI models hallucinate
A large language model is trained to predict the next piece of text. It learns which words tend to follow which, not a database of verified facts. When you ask about something it learned well, the likely answer is usually the right one. When the question touches a gap in its training data, a rare detail or something that happened after its training ended, it still produces the most plausible-sounding text, because that is what it was built to do.
Think of a student who never leaves an exam answer blank: when unsure, they write something that looks like a real answer. A model has no built-in sense of 'I am not sure' unless it has been trained or prompted to express doubt, and it writes errors in the same fluent, confident tone as facts.
Common types of hallucination
The US National Institute of Standards and Technology (NIST) lists this risk as 'confabulation' in its generative AI risk profile, a term that avoids implying the model perceives anything. Researchers also separate output that contradicts the source the model was given from output that adds claims the source cannot support (Ji et al., 2022). In everyday use, the most frequent forms are:
- Invented sources: books, papers, court cases or links that do not exist.
- Wrong specifics: dates, numbers, names, prices or product features.
- Misread documents: a summary that states something the original never said.
- Made-up code: functions, settings or software packages that are not real.
What a hallucination is not
A hallucination is not a lie: the model has no intention to deceive and no awareness that it is wrong. It is also different from outdated knowledge (information that was correct but has since changed) and from bias (a systematic slant learned from training data), although all three can appear in the same answer.
Nor is it a rare glitch that one update will remove. It follows from how these models generate text, so every AI tool can do it, including those that search the web or read your files.
How to reduce hallucinations when you use AI
You cannot switch hallucinations off, but you can make them rarer and easier to catch. When you compare AI tools, look at whether answers show their sources and whether each claim can be traced back to them; that says more about reliability than a general accuracy claim. Practical habits:
- Give the model the source: paste the document, or use a tool that searches trusted sources first (retrieval-augmented generation, or RAG).
- Ask for citations, then open them. A real link or DOI (a permanent identifier for papers) can be checked; a plausible title cannot.
- Tell it that 'I don't know' is an acceptable answer.
- Use a low temperature for factual tasks, and verify numbers, names, legal or medical claims before you rely on them.
Example
Try it: catch a hallucinated citation
Try this promptStep 1. Ask: List three peer-reviewed studies on how coffee affects memory. For each, give the authors, year, journal and DOI. Step 2. Check each DOI by adding it after https://doi.org/ in your browser. Step 3. Ask again with a safer instruction: Only list studies you are confident exist. If you are not sure, say so instead of guessing, and mark any detail you could not verify.
Results vary by assistant and by whether it can search the web. Any DOI that does not resolve, or that leads to a different paper, is a hallucination.
Key points
- A hallucination is confident output that is false or unsupported.
- It comes from predicting likely text, not looking up facts.
- Fake citations, wrong numbers and invented code are the usual forms.
- Grounding answers in sources (RAG) and asking for citations reduces it.
- Always verify facts that matter before you use them.
Try it on Mindber
Related terms
Related topics
Frequently asked questions
Why does ChatGPT make things up?
ChatGPT, like other chat assistants, is built on a language model that generates the most plausible next words rather than retrieving verified facts. When it lacks reliable information, it can still produce a fluent, confident answer. Web search and source citations, where available, reduce the problem but do not eliminate it, so important facts still need checking.
Can AI hallucinations be completely eliminated?
Not given how language models work. Grounding answers in trusted sources, better training, reasoning steps and automatic fact-checks all lower the rate, but any system that generates text by prediction can still produce something false. Treat AI output as a draft from a fast but fallible assistant, and keep human review wherever errors would be costly.
How can I tell if an AI answer is a hallucination?
Look for specifics you can check: names, numbers, quotes, links and citations. Open every source and confirm it says what the answer claims. Be extra careful when an answer is very precise about something obscure, when it cannot show a source, or when asking the same question again produces different facts. Inconsistent answers are a strong warning sign.
Sources
- Survey of Hallucination in Natural Language Generation. Ji et al., arXiv, February 2022
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1. National Institute of Standards and Technology, July 2024
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Lewis et al., arXiv, May 2020
Last updated