What is chunking in AI?
Quick answer
Chunking is splitting long documents into smaller pieces, such as a few paragraphs each, before they are stored for AI search. Each chunk is turned into an embedding, and when a question comes in, only the most relevant chunks are handed to the model to read.
Last updated
Updated · By Robert Breen
Why it matters for a small business
A handbook, a contract file or a year of meeting notes is too long to hand to a model in full for every question, and searching it as one block would match everything and nothing. Chunking cuts it into pieces small enough to search precisely and to fit in the model's context window alongside the question.
How you cut matters. Chunks that are too small lose the sentence that explains them, so the model sees a date without knowing what it is for. Chunks that are too big drag in unrelated text and cost more to process. Tools usually let you set a chunk size and a small overlap between neighboring chunks, so ideas that cross a boundary are not cut in half.
In a real lesson: Reply Faster to a Client Asking for a Case Update: ChatGPT for Small Law Firms
The client case update lesson does not build a document search, but its numbered facts are a hand-made version of good chunks. Carla Mendes, who runs a made-up flower shop, is suing a made-up event venue for $8,400 and wants to know her court date. Owen Park, a paralegal at Harbor Street Law, a made-up firm, pastes nine facts, each one complete on its own.
Look at fact 4. It holds the date and what the date means in one piece: a case management conference on Tuesday, Nov 17, at 9:30 a.m., by video, "a scheduling hearing, not a trial," with no trial date set. If a splitter had cut after the time, a search for "court date" could return the date without the warning, and a reply could tell Carla the judge will hear her case that day.
That is almost what happens in the lesson's first draft, which the check table flags as only partly supported. Even with the full fact pasted, the model stretched it. When you prepare documents for AI search, keep each rule and its exceptions together, the way these facts are written.
Try this lesson free or read the step-by-step guide.
Common confusions
Chunks vs tokens
A token is a tiny unit of text a model reads, often part of a word. A chunk is a passage of many tokens, sized for search. Chunk sizes are often measured in tokens or characters.
Chunking vs summarizing
Chunking keeps the original wording and just divides it. Summarizing rewrites it shorter. Search systems usually store the original chunks so answers can quote the real text.
Tips
- Split on natural breaks like headings and numbered sections where you can.
- Keep a rule and its exceptions in the same chunk.
- Test with real questions and read which chunks come back before trusting the answers.
Related terms
More AI basics terms
Where you use it: free lessons
- Reply Faster to a Client Asking for a Case Update: ChatGPT for Small Law Firms (ChatGPT, 10 min)
- Reply Faster to Claim-Status Emails: ChatGPT for Independent Insurance Agencies (ChatGPT, 10 min)
Prompt templates that use it
Frequently asked questions
- What chunk size should I use?
- There is no single right size. Start with your tool's default, test with questions whose answers you know, and adjust if answers miss context or pull in unrelated text.
- Does n8n do chunking?
- Yes. When you load documents into an n8n vector store, a text splitter node divides them into chunks first, with settings for chunk size and overlap.