How do you spot an AI hallucination in five minutes?

You spot an AI hallucination by stopping at the first specific claim in the answer, asking the tool for its source, opening that source and comparing the sentence on the page with the sentence in the chat, then checking the page date and whether a second page agrees. The routine takes about five minutes and catches the invented number and the citation that leads nowhere at step 3, and the misattributed quote at step 4. I based this on OpenAI's post of 5 September 2025 on why models guess, the EBU and BBC study of 22 October 2025, the Tow Center test of March 2025, Vectara's leaderboard as updated 11 May 2026, the Deloitte Australia case as reported by AP on 7 October 2025, and the claim ledger I keep for my own articles, re-checked on 1 November 2026.

A hallucination is a statement a language model presents as fact with nothing behind it: a number, a quote, a paper title or a link the model produced because it had the right shape. The model keeps no separate record of what it knows, so it cannot flag the invented sentence for you.

The hub on learning AI without coding covers fact-checking in one paragraph; this guide is the long version.

Why does an AI answer contain invented facts?

An AI answer contains invented facts because the model was trained and scored in a way that rewards a confident guess over an admission of not knowing. OpenAI said so on 5 September 2025 in a post titled Why language models hallucinate: "standard training and evaluation procedures reward guessing over acknowledging uncertainty."

The paper behind the post, Why Language Models Hallucinate by Kalai, Nachum, Vempala and Zhang, submitted to arXiv on 4 September 2025, explains it with an exam: "Like students facing hard exam questions, large language models sometimes guess when uncertain." A blank answer scores zero and a guess scores zero or one, so over enough questions guessing comes out ahead. The abstract adds that hallucinations persist "due to the way most evaluations are graded".

OpenAI's post shows the effect with a table from the GPT-5 System Card, on a question set called SimpleQA.

Model	Abstains (says it does not know)	Accurate	Wrong
gpt-5-thinking-mini	52%	22%	26%
OpenAI o4-mini	1%	24%	75%

Source: Why language models hallucinate, OpenAI, 5 September 2025, citing the GPT-5 System Card.

The two models are almost equally accurate, and the one that almost never says "I don't know" is wrong in 75% of its answers, against 26% for the one that abstains in 52% of cases. OpenAI's proposed fix is to "penalize confident errors more than you penalize uncertainty", a change to how models are scored.

How often are AI answers wrong?

AI answers were wrong between 1.8% and 24.2% of the time when a model only had to summarise a document it was given, 45% of the time when assistants answered news questions judged by journalists, and more than 60% of the time when AI search tools were asked to identify a pasted news excerpt. Those are three tests of three different tasks, so read each figure with the task it measured.

Study	What was measured	Result	Published
Hallucination Leaderboard, Vectara	Models summarise a document; the rate is 100 minus the share of summaries consistent with it, scored by HHEM-2.3 "using only the facts presented in the document"	Lowest 1.8% (antgroup/finix_s1_32b), highest 24.2% (ministral-3-3b-2512); gemini-2.5-pro 7.0%, gpt-4o-2024-08-06 9.6%, claude-sonnet-4-6 10.6%, openai/o3-pro 23.3%	Updated 11 May 2026
News Integrity in AI Assistants, EBU and BBC	Journalists from 22 public service media organisations in 18 countries and 14 languages judged more than 3,000 news answers from ChatGPT, Copilot, Gemini and Perplexity	45% of answers had at least one significant issue; 31% serious sourcing problems; 20% major accuracy issues; Gemini had significant issues in 76% of responses	22 October 2025
AI Search Has a Citation Problem, Tow Center at CJR	8 tools, 20 publishers, 10 articles each, 1,600 queries; each tool asked to name the headline, publisher, date and URL of a pasted excerpt	More than 60% of queries answered incorrectly; Perplexity 37% wrong, Grok 3 94% wrong	6 March 2025

The Vectara figures are the narrowest and the easiest to misread. The model has the correct document in front of it and is asked only to summarise it, and it still adds content the document does not contain. openai/o3-pro, a reasoning model, sits at 23.3%, near the top of the table, above gpt-4o-2024-08-06 at 9.6%.

Of the three, the EBU and BBC study is closest to everyday use. Its press release of 22 October 2025 states that "45% of all AI answers had at least one significant issue" and that "31% of responses showed serious sourcing problems", defined there as missing, misleading or incorrect attributions.

The Tow Center test is the one about citations. Premium tools such as Perplexity Pro and Grok 3 answered more queries correctly but "paradoxically also demonstrated higher error rates", because they gave a definite wrong answer where the free versions declined. For Grok 3, 154 of 200 citations led to error pages.

What does the five-minute check look like, step by step?

The five-minute check is five steps with a time limit each, run on the one claim in the answer that would cost you most if it were wrong. I run it before any number, quote or citation leaves my hands, whether for an article or a client summary.

1. Stop at the first specific claim (30 seconds). Pick the number, the quote, the name or the citation you were about to repeat. Pick one claim rather than the whole answer.
2. Ask for the source (30 seconds). Type: "Which page does the 45% figure come from? Give me the URL and the sentence it appears in." A tool with search on gives a link; a tool without search often gives a plausible title. Both get the same next step.
3. Open the source (60 seconds). Click the link, or paste the title into a search engine. If the link errors or the title finds nothing, stop: the claim is unverified and you treat it as absent.
4. Compare the claim with the page (90 seconds). Find the sentence. Check the number, the unit, the population and the time window. "45% of AI answers" and "45% of more than 3,000 news answers judged by journalists, per the EBU release of 22 October 2025" are different claims, and only the second matches the page.
5. Check the date and a second source (90 seconds). Read the page date, then search the claim once more to see whether an independent page reports it or every page repeats one study. If it is one study, say so when you use it.

I ran the five steps on one statistic from this guide.

Claim in the answer	Step 2	Step 3	Step 4	Step 5	Verdict
"45% of AI answers contain errors"	EBU press release, 22 October 2025	Page opens	Page says "45% of all AI answers had at least one significant issue", about news questions judged by journalists	Dated 22 October 2025; the EBU report page confirms the sample of more than 3,000 responses	Use it, reworded to carry the scope and the date

The four quiz questions that drill this routine are in Fact-Checking & Spotting Hallucinations, a beginner module of about ten minutes. The module also covers asking the tool to critique its own answer and defines the two terms it rests on.

How do you make ChatGPT, Gemini, Claude and Perplexity show their sources?

ChatGPT, Gemini, Claude and Perplexity each show sources in a different place, and only Perplexity's help page says every answer carries them. The table comes from the vendor pages I opened on 2 November 2026.

Tool	Where sources appear	Vendor wording	When none appear
ChatGPT	Inline citations in responses that used search; full list under the Sources button at the end of the response	"ChatGPT responses that use search will contain inline citations." (ChatGPT search)	Switch search on, or ask "search the web for this and cite the page", then open every citation
Gemini	A Sources button at the bottom of a response or inline within it, opening a side panel of links	"Not all responses include related links or sources." (View related sources from Gemini Apps)	Ask again with "include the source links"; treat an answer with no Sources button as unsourced
Claude	Anthropic documents citations as an API feature for developers; the chat app's own help pages were not part of this check	"Citations return the exact passages that support each claim" (Citations - Claude Platform Docs)	Ask for the URL and the sentence the claim comes from, and open it yourself
Perplexity	Numbered citations in every answer, each linking to the original page	"Each answer includes numbered citations linking to the original sources" (How does Perplexity work?)	Click every number; a citation whose page does not contain the claim counts as missing

A sources button proves the tool retrieved a page, and the answer can still misstate what that page says. In the Tow Center test, more than half of Gemini's and Grok 3's responses cited fabricated or broken URLs. OpenAI's own page Does ChatGPT tell the truth? says the product "can even make up things like quotes or citations, so don't use it as your only source for research."

When an assistant answers from your own documents through a connector, as covered in the guide on what MCP means in plain words, step 4 becomes opening your own file and finding the line. The job-by-job comparison of ChatGPT and Claude covers which plan on each tool includes search and connectors.

Which signs mean a claim needs checking first?

A specific number with no source, a quote with a name attached, a citation with a title and a year, a URL you have not opened, and a date near or after the model's training cutoff are the claims to check first. The model can produce all five without a page behind them, and they are the sentences that end up in an email or a slide.

Sign	Why the model gets it wrong	What the check does
A specific number with no source	Numbers in training text carry no record of where they came from, so the model recalls the shape and fills in a digit	Steps 2 and 3: ask for the page, open it. If there is no page, do not use the number
A quote with a name attached	A quote only has to sound like the person	Step 4: find the exact words on the page. A paraphrase inside quotation marks fails
A citation with a title, author and year	Citations follow a fixed format, so the model can produce one that is well-formed and empty	Step 3: paste the title into a search engine or the journal's site. Nothing found means it does not exist
A URL you have not opened	The model can build a URL that matches a site's pattern without the page existing	Step 3: click it. If it errors, the claim fails
A date near or after the model's training cutoff	The model has no data for events after its cutoff and completes the pattern anyway	Step 5: check the page date and look for a second, dated source

The routine maps onto a method from 2019 that university libraries now apply to chatbots. Mike Caulfield published SIFT (The Four Moves) on 19 June 2019: Stop; Investigate the source; Find better coverage; Trace claims, quotes and media back to the original context. On time, he wrote that "Taking sixty seconds to figure out where media is from before reading will help you decide if it is worth your time."

Step 1 of the five-minute check is Caulfield's Stop, steps 2 and 3 are Investigate the source, step 4 is Trace, and step 5 is Find better coverage. Boston University Libraries applied SIFT to chatbots in a FAQ dated 7 January 2025, How can I fact-check the information that ChatGPT and other language models give me?: follow the tool's links and confirm the summary matches; if there are no links, search the web and compare several sources.

What would the check have caught in a real case?

The five-minute check would have caught the fabricated references in Deloitte's report for the Australian government at step 3 or step 4, because each one pointed to a paper, a book or a passage that does not exist. As AP reported on 7 October 2025 under the title Deloitte to partially refund Australian government for report with apparent AI-generated errors, the Department of Employment and Workplace Relations had commissioned the report for 440,000 Australian dollars (about US$290,000).

The errors, per AP: a fabricated quote attributed to a federal court judgment, references to nonexistent academic papers, and a nonexistent book attributed to a Sydney University professor. Chris Rudge, a Sydney University researcher in health and welfare law, found them and said the report was "full of fabricated references." Deloitte agreed to repay the final instalment of the contract, and the revised report disclosed that Azure OpenAI had been used. The Register reported on 6 October 2025 that the revision removed "more than a dozen bogus references and footnotes". Neither report I opened stated the refund amount, so I do not state one.

What the report contained (per AP, 7 October 2025)	Step that catches it	How
A quote attributed to a federal court judgment	Steps 3 and 4	Open the judgment, search it for the quoted words, find none
References to nonexistent academic papers	Step 3	Paste the title into a search engine or a journal index; nothing returns
A book attributed to a named professor	Step 3	Search the title with the author's name; nothing returns

What I'd do in your first week

1. Run the check on three answers you already used this week, the ones that went into an email or a slide. Note what step 3 returns for each: a page, an error or nothing.
2. Keep a one-line log: date, claim, the source you opened, and a verdict (matched, reworded, qualified or retired).
3. Run the routine on what leaves your hands, such as a number pasted into a client deck. Skip it for a brainstorm that stays in the chat.
4. On day two or three, do the exercise and the four quiz questions in Fact-Checking & Spotting Hallucinations. The first modules of AI Academy are free; after that the course is EUR 9.99 a month, EUR 69.99 a year or EUR 129.99 once.
5. At the end of the week, count how many logged claims you would have repeated unchecked.

After a month the log looks like the claim ledger I keep for my own articles: 39 claims re-checked against their primary source on 1 November 2026.

Outcome on re-check	Count
Matched the source exactly	20
Rested on a single source; reworded to name it and its date	15
Qualified because the page did not state the basis (for example a price without its billing period)	3
Rested on a single undated compilation	1
Earlier wordings retired because the source page had changed or the figure was no longer on it	4
Sources an automated fetch could not re-open (two of the three confirmed through a second page)	3

The first four rows add up to the 39 claims; the last two count earlier wordings and source pages, not claims among the 39. All 39 had read as correct when I wrote them. Nineteen changed wording or carried a flag after the re-check.

If you use AI answers daily and have never opened a source a chat tool gave you, the fact-check module is the first ten minutes to spend. It sits in AI Foundations, the beginner level of AI Academy, 92 modules in five volumes, and its exercise drills the routine in this guide. Once the check is a habit, the next guide is your first automation in n8n or Zapier, where the same compare-to-source step runs on every output.

Re-check by 3 February 2027: the Vectara leaderboard rows and date, whether the EBU has published a second wave of the news integrity study, and whether the ChatGPT, Gemini and Perplexity help pages still describe their source buttons the same way.