What does top-k mean in RAG?
Quick answer
Top-k retrieval means a search returns the k closest matches, such as the 4 chunks most similar to the question. The AI only sees those chunks. Set k too low and it may miss part of the answer; set it too high and it gets more noise and a longer prompt.
Last updated
Take the free module →Module 14 of the free AI for Customer Support course, about 13 minutes. No account needed.
Updated · By Robert Breen
Why it matters for a small business
In RAG, the AI does not read your whole library for each question. The vector database ranks every chunk by similarity and passes back only the top few. That number, k, is one of the simplest settings to change and one of the most useful when answers come out incomplete or muddled.
A question with two parts, like "do you do business returns and what does it cost?", may need more than one chunk to answer. Raise k and the AI has a better chance of getting both. But every extra chunk adds text the model has to weigh, and loosely related chunks can pull the answer off course. Small numbers such as 3 to 5 are a common starting point for short documents; test before you change it.
In a real lesson: Answer Staff Questions from Your Own Docs using n8n + Pinecone
In Answer Staff Questions from Your Own Docs, the Pinecone Vector Store tool shows Limit set to 4. The lesson explains it plainly: each search brings back the 4 closest chunks. That is top-k with k equal to 4, from the 38 handbook chunks loaded in the previous lesson.
How AI Answers from Your Documents shows why the number matters. The search returns chunks with match scores; the 0.89 chunk with the business return price goes to the AI, while a 0.38 chunk about office hours stays out. The voiceover sums it up: fewer, better chunks give cleaner answers.
Try this lesson free or read the step-by-step guide.
Common confusions
Top-k vs max tokens
Top-k is how many chunks the search returns. Max tokens limits how long the AI's reply can be.
Top-k vs a score threshold
Top-k always returns k results, even weak ones. A threshold drops anything below a set score, however many that leaves.
Tips
- Start with the default Limit of 4 and change it only when testing shows a reason.
- Raise it if answers miss half of a two-part question; lower it if answers wander.
- Tell the AI to say "I don't know" when the chunks don't answer, since top-k still returns something.
Related terms
More AI basics terms
Where you use it: free lessons
- Answer Staff Questions from Your Own Docs using n8n + Pinecone (n8n, 13 min)
- How AI Answers from Your Documents: Retrieval (RAG) Explained (Stepthrough, 7 min)
Frequently asked questions
- What does the k stand for?
- Just a number. Top-k means the top k results, so top-4 returns the four best matches.
- Where is top-k in n8n?
- In the vector store node in a retrieve mode it is the Limit field. The lesson's Pinecone Vector Store tool shows Limit 4.
- Does a higher top-k cost more?
- Usually a little, because more chunk text goes into the model's prompt, and model usage is charged by tokens.