AI Hallucinations Explained: Why AI Makes Things Up and How to Reduce It

An AI hallucination is output that’s presented as fact but isn’t true. The text can be grammatically clean, richly detailed, internally consistent, and delivered with no hedging at all, and still be wrong.

The U.S. National Institute of Standards and Technology (NIST) prefers the term "confabulation" to describe this failure mode. In its Generative AI Profile (published in July 2024), NIST defines confabulation as when an AI confidently gives an answer that’s actually wrong or made up. NIST also points out that words like “hallucination” make it sound like the AI is having a human experience, when it’s really not. The AI isn’t confused, and it isn’t lying on purpose—it just generates text that looks like a good answer based on what it’s seen before.

People keep using the word because it captures how the output feels: vivid, specific, and sometimes untethered from reality.

Here’s a simple example. If you ask a chatbot for the birthday of a lesser-known professor, it may give you a date. If you start a new chat and ask again, you could get a different date. The model may be generating a plausible answer rather than retrieving a verified fact.

Large language models are trained to predict text. Given everything written so far, the model estimates which piece of text most plausibly comes next, then the next, and so on. There’s no fact table underneath, no query against a verified database, no internal flag that says “I have this one” or “I don’t.”

This one design choice is why AI so often makes things up: the model picks what sounds right, not what’s actually true. For instance, saying “The study was published in the Journal of Clinical Oncology in 2019” is easy for the AI, because those words fit together naturally. But the model doesn’t have a way to check if that study is real—it just puts together a sentence that seems plausible.

Several things widen the gap between plausible and true.

Thin or missing training data. Facts that appear once or twice in training, or not at all, have no strong pattern behind them. A 2025 OpenAI paper (Adam Kalai, Ofir Nachum, Santosh Vempala and Edwin Zhang, “Why Language Models Hallucinate”) argues that errors like this are the statistical equivalent of misclassification in ordinary machine learning, not a mysterious glitch. Arbitrary facts with no learnable pattern, like a private individual’s birthday, are exactly where models fail.

Conflicting sources. The internet often contains conflicting information. When training data includes those disagreements, the model may generate the version that best fits the context rather than the version that is actually correct.

Ambiguous prompts. When a question is unclear or too general, the model has to infer what you meant. It can fill in missing context incorrectly and answer a different question from the one you intended.

Implicit pressure to produce an answer. There’s also a subtle pressure for AI to always give you an answer, even when it doesn’t really know. The OpenAI paper points out that current training methods teach models to guess rather than admit uncertainty. Imagine taking a multiple-choice test where you only get points for correct answers—guessing is better than leaving a question blank. Language models learn the same logic. If they say “I don’t know,” it counts as much as being wrong, so they end up giving you a confident-sounding guess instead.

Pattern completion. If a question looks like one with a known shape, the model produces the shape. A request for “the API parameter that controls retries” will usually yield a plausible parameter name whether or not that parameter exists, because the shape of the answer is learned even when the specific content isn’t.

Retrieval failures. If an AI looks things up on the web or in a document before answering, it’s less likely to make stuff up—but it’s not foolproof. Sometimes the search finds the wrong page, skips over what you actually need, or the model misunderstands what it finds. Supplying source material (a process called grounding) helps keep its answers closer to the truth, but it can still wander off track.

Fluency and accuracy come from different things, and the model was optimized for only one.

Human readers use writing quality as a proxy for reliability. We trust text more when it’s detailed, when it carries specific numbers, when it names sources, when the formatting looks professional. Those signals work as heuristics because, among humans, producing them usually takes effort and knowledge. A language model produces all of them for free. A fabricated citation costs it exactly as much as a real one.

NIST puts the risk plainly in AI 600-1: users may act on false content because of the confident nature of the response, or because of the logic and citations accompanying it.

There’s also a length effect. The Stanford RegLab study of commercial legal AI tools found that the product with the higher hallucination rate also produced noticeably longer answers, averaging around 350 words against 219 for its competitor. More sentences mean more checkable propositions, which means more chances to be wrong. A thorough-looking answer isn’t a safer answer. Treat confidence as a property of the writing style, not evidence about the world.

The examples below are illustrative rather than reports of specific incidents, except where a documented study is named.

Fabricated facts

Invented statistics, events, or relationships. Often the most dangerous kind, because there’s nothing structurally odd about the output to trigger suspicion.

Invented sources and citations

You’ll often see AI make up citations that look almost real—like pairing real author names with papers that don’t exist, matching real journal names with the wrong volume number, or listing real court cases but describing the wrong legal outcomes. These kinds of mistakes have actually led to court sanctions. For example, in the 2023 Mata v. Avianca case in New York, a lawyer submitted a legal brief with cases invented by ChatGPT, then asked the chatbot if those cases were real—and it confidently said they were.

Incorrect quotations

A real person, a real book, words they never wrote. Quotes are especially prone to this because the model is generating text that sounds like the source rather than retrieving what the source said.

Made-up URLs

Links that follow a site’s URL conventions perfectly and return a 404.

Wrong dates, names and numbers

Version numbers, founding dates, market sizes, dosages. Numbers are compact and high-confidence-looking, and a single wrong digit is easy to miss.

Nonexistent software features and settings

Step-by-step instructions to click a menu item that isn’t there. Common with fast-moving products whose interfaces changed after training.

Coding hallucinations

Imaginary functions, invented API parameters, and packages that were never published. A study presented at USENIX Security 2025 (Spracklen et al.) generated 576,000 code samples across 16 models and found that at least 5.2% of packages recommended by commercial models didn’t exist, rising to 21.7% for open-source models, with 205,474 unique fabricated package names in total. The researchers note the security implication: an attacker can publish a real malicious package under a name the models repeatedly invent.

Unfaithful summaries

The model is handed a document and still adds claims the document doesn’t contain, or reverses a qualification. Summarization is still generation.

False premises accepted without challenge

You ask “why did Company X discontinue Product Y?” when Company X did no such thing. Many models will answer the question as posed rather than correct you.

Not every wrong answer is a hallucination, and the distinction matters because the fixes differ.

You fix outdated knowledge by giving the model search access or current documents. You often fix a reasoning error by asking it to work through the steps. You fix a hallucination by verifying it against a primary source and by prompting that makes abstention acceptable.

Hallucination risk increases when a question is highly specific, obscure, or depends on one correct fact that appears rarely in the model’s training data. Extra verification is especially useful for:

• Requests for exact quotes or academic references
• Facts about lesser-known people, small companies, or niche products
• Legal provisions, case holdings, and regulatory requirements
• Medical dosing, drug interactions, and diagnostic criteria
• Financial figures, tax rules, and filing deadlines
• Anything from the last few months
• API documentation, parameter names, and library versions
• Statistics quoted to a decimal place
• Historical details of the sort that appear in one paragraph of one book
• Product specifications and feature availability

None of this means avoid AI for these topics. It means the verification effort should scale with the cost of being wrong. Asking a model to explain how a tax concept works is low risk. Asking it for the exact threshold that applies to your filing is not.

Hallucinations remain a known limitation across today’s major general-purpose generative AI systems. The frequency varies by model, version, task, and whether the system is grounded with search or other external sources.

Rates differ by model, by version, by task, and by whether retrieval is in play. What makes cross-product comparison hard is that the public benchmarks measure genuinely different things:

• Vectara’s Hallucination Leaderboard scores how faithfully a model summarizes a document it has been given. It’s a measure of faithfulness to a source, not of world knowledge. Vectara’s updated dataset of more than 7,700 longer documents produces higher rates than its original short-document set, which is why numbers quoted from older articles often look better than current ones.
• Google DeepMind’s FACTS Benchmark Suite, launched in December 2025 with Kaggle, breaks down what it means for an answer to be “factual” into four areas: how well the answer sticks to a given document, what the model knows on its own, how well it can use search, and how it handles questions that include images or other media. Out of 3,513 test questions, the highest score any model got at launch was 68.8%—and none managed to break 70%.
• Artificial Analysis’s AA-Omniscience, released in November 2025, throws 6,000 tough expert-level questions at AI models across six different fields. The scoring system is strict: models lose as many points for a wrong answer as they gain for a right one, and saying **“I don’t know”** doesn’t help or hurt their score. When they first ran the test, only three out of 36 models managed to get more right than wrong. Interestingly, two of the most accurate models still guessed when they didn’t know—64% and 81% of the time! One much smaller model got fewer questions right (just 16% accuracy), but was far more cautious, only guessing 26% of the time. That actually made its answers more trustworthy, because it didn’t pretend to know things it didn’t.

Those three numbers aren’t comparable. A 5% summarization hallucination rate and a 60% knowledge hallucination rate describe completely different tasks. Anyone citing a single percentage as “the hallucination rate” of a model is flattening something that only means anything alongside a benchmark name and a date.

Just because a model is newer doesn’t mean it’s always better. OpenAI’s own report from April 2025 showed that its o3 and o4-mini models actually had higher hallucination rates (33% and 48%) on a particular test than the older o1 model, which had just 16%. Even OpenAI said they needed to look into why that happened. Rankings change with every new release, so if you need to compare models for something important, always check the latest live leaderboards—instead of relying on a single article, even this one.

These are ordered roughly by how much they help relative to effort.

1. Give the model more context. Most hallucinations start with an underspecified question, forcing the model to guess your situation before it answers.

Weak: What’s the best way to structure this?

Better: I’m building a REST API in FastAPI for an internal tool with about 200 users and three endpoints. Data lives in Postgres. What’s a sensible project structure, and what would you need to know to give a better answer?

2. Make uncertainty an acceptable answer. Tell the model explicitly that abstaining is preferred to guessing. Anthropic’s own guidance for reducing hallucinations leads with exactly this: give the model permission to say it doesn’t know.

This helps, and it isn’t a guarantee. In the AA-Omniscience evaluation, every model was instructed before answering that it was better to say so than to get the answer wrong. Most guessed anyway, and often. Treat the instruction as a way to shift the odds, not a switch.

3. Ask for sources, then check them. Requesting citations makes the answer auditable. It doesn’t make the citations real. A fabricated reference is exactly as easy for the model to produce as a genuine one, so a citation you haven’t opened is not evidence.

4. Prefer primary sources. For regulations, read the regulator. For API behavior, read the vendor’s documentation. For a study’s findings, read the study, not a summary of a summary. Primary sources are also the only way to definitively confirm or refute a model’s claim.

5. Separate facts from inference. Ask the model to label which parts of its answer are fact, which are inference, and which are assumptions it made about your situation. The assumptions list is often where the actual problem is hiding.

6. Break complex questions into steps. A question with several dependent parts can fail when one early mistake contaminates everything that follows. Breaking the task into smaller steps makes each part easier to verify and helps catch errors earlier. This is most useful for multi-step reasoning and calculations, and less useful for a single obscure fact.

7. Supply the source material yourself. If you copy and paste the actual contract, policy, or documentation and ask about that text, you’ll usually get a more accurate answer than if you just ask the AI to remember what it learned during training. When dealing with long documents, ask the model to pull out the exact sections it’s using before it starts analyzing—this way, you can see where its answers are coming from. This method, called grounding, cuts down on mistakes a lot, but it doesn’t get rid of them completely. That’s exactly what benchmarks for summarization are designed to measure.

8. Turn on search when freshness matters. When you need up-to-date information, retrieval is usually more reliable than asking the model to rely on learned knowledge alone. It is still not foolproof: search may return an irrelevant or low-quality page, the model may cite a source that does not support its claim, or it may blend retrieved material with older assumptions from training. DeepMind designed a search-specific factuality benchmark for exactly this problem, and scores remained well below perfect.

9. Watch for agreement with your premises. If you ask a leading question, you’ll often get a confirming answer. Try asking the opposite of what you believe and see whether the model switches sides. If it agrees with both, it doesn’t know.

10. Start a fresh conversation when the thread goes bad. If a mistake slips into your chat with an AI, it can stick around—future answers might keep using that wrong information as if it’s true. Trying to fix things in the middle of a conversation sometimes helps, but other times the error lingers in the background. If you’re dealing with anything important, it’s safest to start fresh with a brand new question.

1. Mark the checkable claims. Go through and underline every name, date, number, quote, law, version, and citation you see. Don’t worry too much about opinions or explanations—what really matters (and what can get you in trouble) are the specific facts.
2. Rank them by consequence. What actually goes wrong if this item is false? Check the top two or three first. You’re not verifying an essay; you’re verifying the load-bearing claims.
3. Open the source; don’t just confirm it exists. A real paper with the wrong finding attached is the most common failure and the easiest to miss. Search the document for the specific claim.
4. Cross-check consequential claims against a second independent source. Two outlets reprinting the same press release count as one source.
5. Ask the model which parts it’s least sure of. Self-reported uncertainty is imperfect, but it’s cheap, and it surfaces weak spots more often than not.
6. Fix the prompt, not just the answer. If the error came from a bad assumption, correcting the assumption and re-running beats patching the output.

For low-stakes work, steps one and two are usually enough. For anything published, filed, or acted on, do all six.

Uncertainty labeling

Only answer questions when you’re pretty sure about the facts. If something feels uncertain, call it out, and be honest when you just don’t know the answer. I’d much rather have a shorter reply with a few blanks than a long answer that’s full of guesses.

Verification plan instead of citations

For each factual claim in your answer, name the specific source a person should check to verify it, and say whether you’re confident that source contains the claim. Don’t produce citations you aren’t sure exist.

Document-only answering

Answer only from the document below. If the answer isn’t in it, say the document doesn’t cover it. Quote the exact lines you’re relying on before you interpret them.

Assumption audit

Before answering, list every assumption you’re making about my situation that I haven’t told you. Then answer, and mark which parts depend on those assumptions.

Adversarial check on an answer you already have

Here’s an answer you gave me. Argue against it. What’s most likely to be wrong, what would you need to check, and what would change your conclusion?

Premise check

Before answering, tell me whether the question contains any false or unverifiable premises. If it does, correct them instead of answering as asked.

Confidence sort

Split your answer into three lists: things you’re confident about, things you believe but would want checked, and things you’re guessing. Put nothing in the first list you wouldn’t stake a reputation on.

Reduced, clearly yes. Eliminated, no current system has demonstrated that consistently, and the distinction matters.

The OpenAI paper makes a useful distinction. Perfect accuracy isn’t achievable, because some questions are genuinely unanswerable from available information, no matter how large the model or how good the search. But hallucination specifically isn’t inevitable, because a model always has the option of abstaining. The authors argue the fix is partly social rather than technical: as long as the benchmarks that rank models award nothing for “I don’t know,” labs will keep shipping models that guess.

A few key advances are helping bring down the number of AI hallucinations. When an AI system can look up information (“retrieval”) instead of just guessing, its answers are more likely to be accurate. Giving models tools—like letting them run code or query a database—makes them less likely to invent details. New methods for checking the model’s reasoning or double-checking claims against real sources also catch errors before they reach you. And now, some benchmarks are rewarding models for being well-calibrated (knowing when they’re unsure), not just for getting the most right answers—so labs are starting to train for reliability, not just raw accuracy.

Claims that AI hallucinations can be completely eliminated deserve careful scrutiny against real-world evidence. LexisNexis, for example, advertised its legal AI as “hallucination-free.” Thomson Reuters said its tools didn’t have this problem because they relied on trusted sources. But when researchers at Stanford actually put these products to the test, they still found hallucination rates between 17% and 33%—compared to about 43% for general-purpose GPT-4 on the same tasks. That’s a big improvement, but it’s not a full solution.

Reducing hallucinations and eliminating them are different problems. Progress on the first is measurable and ongoing. The second would require a system that reliably recognizes the limits of its own knowledge, something no current system has demonstrated consistently across broad real-world use.

A wrong restaurant recommendation and a wrong drug interaction aren’t the same problem, and treating them the same way wastes effort in one direction and invites harm in the other.

Low stakes. Brainstorming, first drafts, rewriting, naming things, explaining a concept you’ll encounter again. Errors are cheap and usually self-correcting. Verification: skim it.

Medium stakes. This covers things like research you plan to use in future work, market insights, technical advice you’ll actually follow, or anything that will end up in a report for others to read. Mistakes here can waste your time or even hurt your reputation. It’s worth double-checking the details that matter before you rely on them.

High stakes. When it comes to medical, legal, financial, safety, regulatory, or major business decisions, the stakes are high. If you get something wrong, it can be expensive—or you might not be able to fix it at all. That’s why you should always check important details with reliable sources, compare what you find, and reach out to a real expert if possible. Treat anything an AI tells you as a starting point, not the final word.

The useful question isn’t “is AI reliable?” It’s “what happens if this specific claim is wrong, and who bears that?”

A tool doesn’t have to be perfectly reliable to be worth using. Search engines return bad results, reference books contain errors, and experts are sometimes wrong. What makes those tools usable is that most people have a working sense of how they fail.

What’s tough about using generative AI is that its mistakes are often invisible—even glaring errors can look perfectly convincing. Wrong answers sound just as confident and polished as the right ones, because the AI uses the same approach for both. Once you understand that a smooth-sounding answer isn’t necessarily a true one, you can use AI more safely: ask it to flag when it’s unsure, use it for things it does well, and always double-check anything that really matters.