Hallucination

A language model predicts the most probable continuation of a prompt based on patterns in its training data. When it lacks the relevant knowledge, or the prompt pushes it toward an answer, it can still produce fluent text in the same confident tone. Researchers distinguish between output that contradicts the source material and output that cannot be verified from it.

Why it matters: hallucinations are the main reason AI output needs review, especially in legal, medical, financial and academic work, where invented citations or figures can cause real harm. When comparing tools, look for features that reduce and expose the risk: grounding in your documents or search results (RAG), citations you can click, and clear signals when the model is uncertain.

Example: asked for case law supporting an argument, a model may return a realistic-looking case name and citation that does not exist.

Limits of the fixes: retrieval, better prompts and newer models reduce hallucinations but do not eliminate them. Treat AI output as a draft and check claims that matter against the original source.