AI hallucinations: why ChatGPT makes things up, and how to catch it

1. 1

   What is an AI hallucination?

   OpenAI defines hallucinations as “plausible but false statements generated by language models.” The key word is plausible. A hallucination does not look like an error. It looks like a normal, confident answer.

   OpenAI’s own researchers gave an example from their work. When a popular chatbot was asked for the PhD thesis title of one of the authors, it gave three different titles, all wrong. Asked for his birthday, it gave three different dates, also all wrong.

   • Made-up books, research papers or court cases
   • Wrong dates, numbers and statistics stated precisely
   • Links to web pages that never existed
   • Summaries that add details not in the original
2. 2

   Why does AI make things up?

   Remember from the LLM lesson that a model writes by predicting the next word. During training it sees huge amounts of text, but nothing labels each sentence as true or false. Patterns like grammar get learned well. Rare, one-off facts, like a particular person’s birthday, cannot be worked out from patterns, so the model fills the gap with something that sounds right.

   OpenAI’s 2025 research adds a second reason: models are often scored on accuracy alone, like a multiple-choice exam with no negative marking. Guessing sometimes earns points, while “I don’t know” always earns zero. So models learn to guess.

                         gpt-5-thinking-mini   o4-mini (older)
   Said "I don't know"          52%                  1%
   Right answers                22%                 24%
   Wrong answers                26%                 75%
   
   The older model got 2 points more right answers by guessing,
   but almost three times as many wrong ones.

   Key idea

   A model that never says “I don’t know” is not more accurate. It is just guessing more.

3. 3

   A real example: the lawyer and the fake cases

   In 2023, in a US case called Mata v. Avianca, a New York lawyer filed a court brief that cited six earlier court decisions, including one called “Varghese v. China Southern Airlines”. None of them existed. ChatGPT had made them up, with realistic names and citation numbers.

   The lawyer had actually asked ChatGPT, “Is Varghese a real case?” It said yes and claimed the cases could be found in legal databases like LexisNexis and Westlaw. On 22 June 2023, Judge P. Kevin Castel fined the lawyers and their firm $5,000 and ordered them to write to each real judge who had been falsely named as an author of a fake opinion.

   Key idea

   Asking the same AI “are you sure?” is not checking. Checking means an independent source.

4. 4

   How to spot a hallucination

   You cannot tell from the tone. Hallucinations are written in the same confident style as correct answers. Instead, be extra careful in the situations where they are most common.

   One useful test from Anthropic: ask the same question again in a fresh chat. If the answers disagree on a fact, at least one of them is invented.

   • Very specific details: names, dates, numbers, quotes, citations
   • Niche topics: your small town, a small company, a recent rule change
   • Links: click every one before you trust or share it
   • Answers that change when you ask again in a new chat
   • Anything with real stakes: health, money, law, exam facts, your resume
5. 5

   How to reduce hallucinations

   Anthropic’s guide lists simple prompt habits that cut down made-up answers. The biggest one is to give the model permission to say it does not know. Next, give it the actual source (a PDF, a web page, your notes) and tell it to answer only from that, quoting the parts it used.

   For current facts, use a tool that searches the web and shows sources, like Perplexity or the web search option in ChatGPT, Claude or Gemini, and then open those sources. Anthropic is clear that these methods reduce hallucinations but do not remove them, so always check anything important.

   Weak: What is the minimum attendance rule for my university’s exams? Strong: <rules> [paste the text of your university’s attendance rules here] </rules> Using only the rules above, what is the minimum attendance needed to sit the end-semester exam? Quote the exact line you used. If the rules don’t say, reply "Not stated in the document" instead of guessing.

   Key idea

   Give it the source, let it say “I don’t know”, ask for quotes, then verify.

From the people who built it

Straight from the source

The original idea, then the same idea taken apart, then how you use it today.

Adam Kalai, Santosh Vempala and colleagues at OpenAI

OpenAI researchers, working with Vempala, a computer scientist at Georgia Tech, who studied where hallucinations come from and why they persist.

Why language models hallucinate (OpenAI research, September 5, 2025)

1 · Understand

They argue hallucinations are not a mysterious glitch. Pretraining cannot learn rare, one-off facts from patterns, and later testing that scores only accuracy rewards a confident guess over an honest “I don’t know”.

2 · Break it down

• Patterns vs one-off facts: spelling follows rules a model can learn, but a person’s birthday does not, so the model fills the gap with a guess.
• Exams with no negative marking: if a wrong answer costs nothing, guessing always beats leaving it blank. Models are mostly scored this way.
• Their fix: “Penalize confident errors more than you penalize uncertainty”, and give partial credit for honest uncertainty.
• Not inevitable: a model can avoid many hallucinations by declining to answer when it is unsure.

3 · Use it

Tell the chatbot that “I’m not sure” is an acceptable answer, and value it when you get one. A model that sometimes admits doubt is often more trustworthy than one that always has an answer.

5 terms

Words to know

Hallucination

A confident, plausible-sounding AI answer that is false or made up.

Grounding

Giving the model real source material (documents, search results) and asking it to answer from that.

Citation

A pointer to where a claim came from, such as a quote, page or link, so you can check it.

Abstain

When a model declines to answer and says it does not know, instead of guessing.

Verification

Checking a claim against an independent, trusted source, not by asking the same AI again.

1 video · plays here

Watch

Hand-picked from the people who explain this best. The video loads only when you press play.

Hands-on · 10 minutes or less

Try it

Go on a hallucination hunt

1. 1Open any free chatbot and turn off web search if it has a toggle.
2. 2Ask about something niche you know well, such as your school’s founding year, a local shop’s history or a small company you worked at. Note any confident but wrong details.
3. 3Ask: “Give me 3 research papers on [a narrow topic from your field], with authors, year and links.” Search each title on Google Scholar and see which exist.
4. 4Ask the same question again in a new chat. Do the answers match?
5. 5Now repeat with: “If you are not sure, say so. Only list papers you are confident exist.” Compare the results.

Check papers on Google Scholar

3 questions · instant feedback

Check yourself

Pick an answer to see if you are right, and why.

Question 1

Question 1 of 3

Why do language models hallucinate rare facts like a person’s birthday?

AThey copy them from fake websites on purposeBSuch facts cannot be predicted from language patterns, so the model fills the gap with something plausibleCThe free plan blocks correct answersDThey only happen when the internet is slow

Question 2

Question 2 of 3

In Mata v. Avianca, the lawyer asked ChatGPT if a case was real and it said yes. What is the lesson?

AUse a paid version and it will be rightBAsk the question twice to be sureCVerify with an independent source; the same AI cannot confirm its own claimsDNever use AI for anything

Question 3

Question 3 of 3

Which prompt is most likely to reduce made-up answers about a company policy?

A“Tell me the leave policy.”B“You are an expert. Tell me the leave policy accurately.”C“Using only the policy text below, answer and quote the line you used. If it is not there, say so.” (with the text pasted)D“Answer quickly and confidently.”

Lesson 14 of 36

Done? Keep going

Mark this lesson complete to keep your place. Progress is saved on this device, no login.

Mark lesson complete

Up next · Lesson 15 · 11 minChatGPT vs Claude vs GeminiAs of November 2026, ChatGPT, Claude and Gemini are all strong, all have free plans, and their main paid plans cost about $20 a month in the US. None is best at everything, so choose by task: Gemini if you live in Google apps, ChatGPT for the widest set of built-in tools, Claude for long writing and coding work, and Perplexity when you need sources. The best test is to run your own task in two of them and compare.← Previous: Context engineering

People also ask

Questions people ask

What is an AI hallucination?+

It is when an AI chatbot gives a confident, believable answer that is false, such as a fake citation, a wrong statistic or a link that does not exist. OpenAI calls them “plausible but false statements”.

Why does ChatGPT make up fake sources?+

It writes by predicting likely text, so a citation that looks right can be generated even if it does not exist. Training that rewards guessing over saying “I don’t know” makes this worse. Always click and check sources.

Can you stop AI from hallucinating completely?+

No. You can reduce it a lot by giving the AI your own documents, telling it to answer only from them with quotes, and allowing it to say it does not know. But you still need to verify anything important.

7 sources · checked Nov 2026

Sources and further reading

• OpenAI: Why language models hallucinate (Sept 2025)
• Anthropic: Reduce hallucinations
• Google Cloud: What are AI hallucinations?
• Mata v. Avianca, Inc., Opinion and Order on Sanctions, S.D.N.Y., 22 June 2023 (Justia)
• Hinshaw & Culbertson: New York lawyer who cited non-existent cases generated by ChatGPT
• Wikipedia: Mata v. Avianca, Inc.
• Perplexity Help Center: What is Perplexity?

Related on this site

Previous: Prompting techniques that workNext: ChatGPT vs Claude vs GeminiHow LLMs workEmbeddings and RAGAI engineer guideFree AI skills resources