# Document Bot Mistakes: 7 Reasons a RAG Bot Answers Wrong

> Seven reasons a bot that answers from your documents gets it wrong: old prices in files, scanned PDFs, files that disagree and no fallback rule.

Source: https://learn.ynteractive.com/content/blog/ai-retrieval-bot-mistakes · Updated 2026-10-20 · Free from Stepthrough (https://learn.ynteractive.com)

Quick answer

When a bot that answers from your documents gets something wrong, the cause usually starts with the files, not the AI. Old prices left in a file, a [scanned PDF](https://learn.ynteractive.com/content/glossary/searchable-pdf) with no selectable text, two files that disagree, or no rule telling the bot what to do when the files don't cover a question. The fix order is the same every time: fix the source first, then the instructions. The free [How AI Answers from Your Documents (Retrieval, Explained)](https://learn.ynteractive.com/modules/ai-retrieval-explained) lesson walks one question through a support bot so you can see where each mistake creeps in.

Last updated October 20, 2026

[Take the free module →](https://learn.ynteractive.com/courses/build-an-ai-support-bot/3)Module 3 of the free [Build Your Own AI Support Bot](https://learn.ynteractive.com/courses/build-an-ai-support-bot) course, about 7 minutes. No account needed.

By [Robert Breen](https://learn.ynteractive.com/content/about) · Published October 20, 2026 · Updated October 20, 2026 · 4 min read

AI Basics · RAG · AI Support Bot · Troubleshooting

The example business is Alder & Finch Accounting, a made-up five-person firm with three files behind its website bot: a client FAQ, a fee sheet and an engagement policy.

## What are the most common retrieval bot mistakes?

| # | Mistake | Fix |
| --- | --- | --- |
| 1 | Old prices or policies left in a file | Remove them before you upload |
| 2 | Editing a file but not uploading it again | Re-upload after every change |
| 3 | Scanned PDFs with no selectable text | Run [OCR](https://learn.ynteractive.com/content/glossary/ocr) first, or use a text file |
| 4 | Two files that disagree | Keep each answer in one current file |
| 5 | No "answer only from the sources" rule | Put it in the instructions |
| 6 | No [fallback answer](https://learn.ynteractive.com/content/glossary/chatbot-fallback-response) for weak matches | Give one that points to a person |
| 7 | Testing only questions the bot can answer | Add one question it shouldn't know |

## Why do the files cause most bad answers?

**1. Old prices or policies in a file.** Retrieval reads your files on every question, and the bot quotes whatever is there. If last year's fee sheet is still in the [knowledge base](https://learn.ynteractive.com/content/glossary/knowledge-base), the bot can find it and quote it. Remove old prices and policies before you upload.

**2. Editing without re-uploading.** When you add a file, the tool cuts it into chunks and keeps them in a search index. Change a price in your copy of the file and the index still holds the old chunk. Upload it again, or the bot keeps the old answer.

**3. Scanned PDFs.** A scanned PDF is a picture of text. The bot sees nothing in it. Use files with selectable text, or run [OCR](https://learn.ynteractive.com/content/glossary/ocr) first. The [OCR and document parsing FAQ](https://learn.ynteractive.com/content/blog/ocr-document-parsing-faq) explains the difference.

**4. Files that disagree.** If the FAQ says one thing and the policy says another, either chunk can match a question. Before you upload, list the questions customers ask most and check each answer is written down in one current file.

## Try it free, step by step

**How AI Answers from Your Documents: Retrieval (RAG) Explained**. Follow one customer question through a support bot: files cut into chunks, a search by meaning, the AI told to answer only from what it found, and a safe fallback when nothing matches. You do every step yourself in a practice copy of Stepthrough, it checks your work as you go, and “Do it for me” finishes any step you get stuck on. Free, and nothing touches your real accounts.

[Start the free lesson](https://learn.ynteractive.com/modules/ai-retrieval-explained)

[Or read the full step-by-step guide](https://learn.ynteractive.com/content/how-ai-answers-from-your-documents)

## Which instruction mistakes let the AI make things up?

**5. No source rule.** The AI receives three things: instructions, the matching chunks and the question. The instruction "answer only from the sources, say so if they don't cover it" is the whole point. Without it, the AI fills gaps from memory, and that is where wrong prices come from. [Grounding](https://learn.ynteractive.com/content/glossary/grounding) and [hallucination](https://learn.ynteractive.com/content/glossary/hallucination) explain why.

**6. No fallback answer.** In the lesson, a visitor asks about taxes for a crypto mining rig. None of the three files covers it, and every match is weak (0.31, 0.27 and 0.22). The bot says it isn't sure, doesn't want to guess, and gives Nora's email, phone and office hours. Weak matches are the signal to stop. A plain chatbot with no documents would have answered confidently anyway. The [support bot guardrails FAQ](https://learn.ynteractive.com/content/blog/ai-support-bot-guardrails-faq) shows how to write the fallback and the rules around it.

## What testing mistake hides all the others?

**7. Testing only the easy questions.** A bot that answers your five most common questions well can still guess on the sixth. Before a bot goes live, ask five real questions and one it shouldn't be able to answer. If it guesses on that one, tighten its rules before anyone else sees it.

A related mix-up is thinking the bot was "trained" on your files. It wasn't. Training ([fine-tuning](https://learn.ynteractive.com/content/glossary/fine-tuning)) changes the model and is slow to update. Retrieval reads your current files on every question. That is good news: you fix an answer by fixing the file.

## Key takeaways

- Bad answers usually start with the files. Fix the source first, then the instructions.
- Remove old prices, re-upload after every edit, and use files with selectable text.
- Keep each answer in one current file so files don't disagree.
- "Answer only from the sources" plus a fallback answer stops made-up replies.
- Always test one question the bot shouldn't know.

More answers are in the [retrieval FAQ](https://learn.ynteractive.com/content/blog/ai-retrieval-rag-faq). To build a real one, see [Chatbase questions answered](https://learn.ynteractive.com/content/blog/chatbase-questions-answered).

## Free lessons in this post

- [How AI Answers from Your Documents: Retrieval (RAG) Explained](https://learn.ynteractive.com/content/how-ai-answers-from-your-documents): Follow one customer question through a support bot: files cut into chunks, a search by meaning, the AI told to answer only from what it found, and a safe fallback when…

## Related terms

Plain-English definitions, each with an example from a free lesson.

[Searchable PDF](https://learn.ynteractive.com/content/glossary/searchable-pdf) · [OCR (optical character recognition)](https://learn.ynteractive.com/content/glossary/ocr) · [Chatbot fallback response](https://learn.ynteractive.com/content/glossary/chatbot-fallback-response) · [Chunking](https://learn.ynteractive.com/content/glossary/chunking) · [Cosine similarity](https://learn.ynteractive.com/content/glossary/cosine-similarity) · [Embeddings](https://learn.ynteractive.com/content/glossary/embeddings) · [Grounding](https://learn.ynteractive.com/content/glossary/grounding) · [Knowledge base](https://learn.ynteractive.com/content/glossary/knowledge-base)

[The full AI glossary](https://learn.ynteractive.com/content/glossary)

## Keep reading

- [How AI Answers from Your Documents: Retrieval FAQ](https://learn.ynteractive.com/content/blog/ai-retrieval-rag-faq): How does a support bot answer from your own files? Chunks, embeddings, a search by meaning, "answer only from the sources" and a safe fallback, explained.
- [Speech-to-Text Mistakes: 7 Transcript Errors to Catch](https://learn.ynteractive.com/content/blog/speech-to-text-mistakes): Seven mistakes with AI transcripts: bad audio, unchecked names and numbers, unnamed speakers, swapped voices, unchecked action items and no consent.
- [Document AI Mistakes: 7 Ways OCR and Extraction Go Wrong](https://learn.ynteractive.com/content/blog/ocr-document-ai-mistakes): Seven mistakes when AI reads invoices and PDFs: skipping OCR on scans, trusting scanned numbers, rules on changing layouts, guessed fields and no checks.

[All blog posts](https://learn.ynteractive.com/content/blog) · [All free tutorials](https://learn.ynteractive.com/content)