What is an AI hallucination?
Updated · By Robert Breen
An AI hallucination is when a model states something false, made up or unsupported as if it were fact. It can be an invented price, a policy you never had, a fake source or a promise you cannot keep, and it usually sounds just as confident as the true parts.
Why it matters for a small business
Language models are built to produce likely-sounding text, not to check facts. When they lack information, they often fill the gap with something plausible instead of saying "I don't know". In casual use that is an annoyance. In a customer email, an estimate or a client update, it can mean a refund you did not authorize, a guarantee you cannot honor or a safety claim that is simply wrong.
You cannot switch hallucinations off, but you can make them rare and catch the ones that slip through. Give the model the facts it is allowed to use, tell it not to invent anything beyond them, ask it to check its own draft against those facts, and read the result yourself before it goes out. The person sending the message is still responsible for what it says.
In a real lesson: Reply Faster: Turn a Messy Customer Email Thread into a Clear, Friendly Reply
In the Reply Faster lesson, you answer Dana Whitfield at Cedar Lane Bakery, a made-up neighborhood bakery. Her cake order has the wrong pickup day, a switch to lemon and an unanswered question about her sister-in-law's nut allergy. You paste the thread, then click Paste facts. Fact 4 is careful: the lemon recipe has no nuts, but the kitchen also bakes with nuts, "so we can't promise any cake is nut-free."
The first draft reads well and still says the cake will be completely nut-free. That line is a hallucination with real consequences: it sounds caring, matches what Dana wants to hear and is not in your facts.
So you click Paste check, asking for a table of every promise in the reply and the fact that backs it up, with anything unsupported flagged. Five of six claims match; the allergy line does not, and ChatGPT suggests honest wording. Paste fix applies it and keeps every other fact the same, and you still read the final reply before clicking Send.
Try this lesson free or read the step-by-step guide.
Common confusions
Hallucination vs a mistake from bad input
If you paste the wrong price, the model repeating it is not a hallucination; it did what you asked. A hallucination is content the model added that your input does not support. The fix for one is checking your facts, for the other grounding the model in them.
Does a confident tone mean it is true?
No. Models write true and invented statements in the same assured voice. Judge a claim by whether a source or your facts support it, not by how certain it sounds.
Tips
- Number your facts so you and the model can match each claim to one.
- Add the rule "Don't invent details or promise anything that isn't in the facts" to any prompt that writes to customers.
- Watch hardest at dates, prices, guarantees, allergies, legal outcomes and anything a customer will rely on.
Related terms
Where you use it: free lessons
- Reply Faster: Turn a Messy Customer Email Thread into a Clear, Friendly Reply (ChatGPT, 9 min)
- Reply Faster to Clients Chasing Their Refund: ChatGPT for Tax and Bookkeeping Firms (ChatGPT, 10 min)
- Reply Faster to Claim-Status Emails: ChatGPT for Independent Insurance Agencies (ChatGPT, 10 min)
Prompt templates that use it
Frequently asked questions
- Why does AI make things up?
- It generates the most likely next words based on patterns, and it has no built-in fact check. When it lacks the real answer, a plausible guess can come out sounding certain.
- Can I stop ChatGPT from hallucinating completely?
- No, but you can reduce it a lot by supplying the facts, forbidding invented details and asking it to check its draft against your facts. A final human read catches the rest.