The example is Alder & Finch Accounting, a made-up five-person firm that wants its website bot to answer from its own files, not from whatever the AI half remembers.
What happens when you upload your files?
Three steps, once, before anyone asks a question. Alder & Finch uploads three files: Client FAQ.pdf (22 questions and answers), Fees 2026.docx (one page) and Engagement policy.txt (two pages, covering refunds, deadlines and services the firm doesn't offer).
- Split. Each file is cut into chunks of a few sentences. A 3-page file becomes about 20 small pieces.
- Index. Each chunk is turned into an embedding, a list of numbers that captures what it means. "What do you charge?" and a line from the fee sheet end up close together, even with no words in common.
- Store. The chunks and their numbers sit in a search index, often a vector database. Pinecone is one; tools like Chatbase keep one for you.
What happens when a question comes in?
The visitor asks: "Do you prepare business tax returns, and what does it cost?" The bot searches the index first and scores the closest chunks by meaning:
| Chunk | Match | Used? |
|---|---|---|
| Fees 2026, part 2 (business return fees) | 0.89 | Yes, answers the cost half |
| Client FAQ, question 7 ("Yes, we prepare business returns, forms 1120-S and 1065.") | 0.84 | Yes, answers the first half |
| Client FAQ, question 2 (opening hours) | 0.38 | No, below the cut-off |
The AI then receives three things: the instructions ("answer only from the sources, say so if they don't cover it"), the two matching chunks with their file names, and the visitor's words. The answer names its sources: Client FAQ, question 7, and Fees 2026.
Try it free, step by step
How AI Answers from Your Documents: Retrieval (RAG) Explained. Follow one customer question through a support bot: files cut into chunks, a search by meaning, the AI told to answer only from what it found, and a safe fallback when nothing matches. You do every step yourself in a practice copy of Stepthrough, it checks your work as you go, and “Do it for me” finishes any step you get stuck on. Free, and nothing touches your real accounts.
Start the free lessonFrequently asked questions
- What is retrieval-augmented generation (RAG)?
Find the right pieces of your documents, then have the AI write the answer from them, and show where it came from. The retrieval step is the search. The generation step is the AI writing.
- Why does the bot quote my real fee instead of a typical one?
Because it found the matching chunk of your fee sheet and was told to answer only from it. It wasn't retrained on your website, and it didn't search the web.
- Why doesn't it send the AI every file?
Fewer, better chunks give cleaner answers. Only the best few go to the AI, not all 22 FAQ questions. Weak matches, like the opening-hours chunk at 0.38, stay out.
- What is the most important instruction?
"Answer only from the sources." The lesson calls this rule the whole point. Without it, the AI fills gaps from memory, and that is where wrong prices come from. Grounding is the general idea.
- What happens when my files don't cover a question?
Every match is weak, and the bot gives a fallback answer. In the lesson, a visitor asks about taxes for a crypto mining rig. The best match scores 0.31, so the bot says it isn't sure, doesn't want to guess, and gives Nora's email and phone number with office hours. A plain chatbot with no documents would have answered confidently anyway. See chatbot fallback response.
- Is retrieval the same as training the AI on my files?
No. The lesson compares three things. Pasting a file into one chat only helps that chat. Training (fine-tuning) changes the model and is slow to update. Retrieval reads your current files on every question, so you fix an answer by fixing the file.
- I changed a price in my file. Why does the bot still quote the old one?
The index still holds the old chunk. Change a file and you upload it again, or the bot keeps the old answer.
- Where will I see retrieval in real tools?
Under different buttons. Chatbase calls your files Data sources. tawk.to builds its AI agent from your website. In n8n you load a Pinecone index yourself. Under the hood, all of them retrieve, then answer.
- Why does the bot see nothing in my PDF?
It may be a scanned PDF with no selectable text. That needs OCR first. The OCR and document parsing FAQ explains how.
- How do I test a bot before it goes live?
Ask five real questions and one it shouldn't be able to answer. If it guesses on that one, tighten the fallback before anyone else sees it.
- Do I need to build these steps myself?
Usually not. In Chatbase you upload a file and the split, index and store steps happen for you. In n8n with Pinecone, you build each step yourself.
Key takeaways
- A retrieval bot searches your files first, then answers only from what it found.
- Files become chunks, chunks become embeddings, and embeddings sit in a search index.
- "Answer only from the sources" is the rule that stops made-up answers.
- Weak matches mean "I don't know", followed by a way to reach a person.
- Fix a wrong answer by fixing the file, then upload it again.
See what usually goes wrong in retrieval bot mistakes, then start the free lesson.