Learn AI /Search, Embeddings and RAG /Chunking
Chunking
- Intermediate
- Search, Embeddings and RAG
- Technical readers
- Programme & data teams
Also called: chunk, chunks, splitting.
Back to the milestone you came from
In five seconds
What it is
Chunking splits source material into smaller pieces so search and retrieval can find and return the most relevant passage, rather than an entire document or a random fragment.
In plain words
Take a long document, break it into sensible pieces, and search those pieces instead of the whole document. The goal is to bring back the passage that best answers the question, with enough surrounding context to make sense.
An everyday example
At a port warehouse in Mombasa, Amina receives a 40-page shipping manifest and needs to check whether a container matches what was delivered. Instead of searching the whole stack at once, she places sticky tabs to divide it into sections such as electronics, medical supplies and spare parts, with a few repeated lines near section breaks so item descriptions are not split awkwardly. When a box marked "water pump gasket" arrives, she goes straight to the likely section and then to the nearest item list, rather than rereading all 40 pages. The task becomes faster because she searches smaller, meaningful sections, not one giant document.
A manifest already has human-made sections and exact item names, while real documents may have weak structure and a search system often matches by meaning, not only exact words. Also, a person can use background knowledge to jump around flexibly, whereas chunking rules are mechanical unless extra logic is added.
Remember thisChunking determines the units a search system can retrieve. Chunks that are too large dilute relevance; chunks that are too small can separate a statement from the context that gives it meaning.
Understand it
Understand it
Why it matters
Non-specialists often judge a retrieval system by the final answer, but chunking strongly shapes whether the right evidence can be found at all. If chunks are too large, search may pull in broad but unfocused text. If they are too small, important context may be missing. This affects answer quality, citation quality, cost, and review effort. For managers and programme staff, chunking is therefore not a minor technical setting. It influences whether a system is useful for policy lookup, guidance drafting, question answering and auditability. When results seem inconsistent, chunking is one of the first design choices worth checking.
How chunking helps retrieval find the right passage
Show how one long document becomes smaller retrievable passages, and why chunk size and overlap affect what can later be found.
01
1. Long source
Begin with a single long document such as a report or guidance note. Important ideas may be far apart inside it.
Draw one tall document card with multiple paragraph lines and section headings.
02
2. Split into chunks
Cut the document into smaller pieces. Boundaries can follow length rules, paragraphs or headings.
Show the document becoming four horizontal cards labelled Chunk A, B, C and D.
03
3. Add overlap if needed
Repeat a small amount of text across neighbouring chunks so context is not lost at the boundary.
Show Chunk B and C with a shared shaded strip at their touching edges; add a note 'overlap'.
04
4. Index each chunk
Store each chunk separately for later search, often with an embedding and metadata such as title or page.
Connect each chunk card to a storage panel with rows representing indexed records.
05
5. Retrieve relevant passages
A question is compared with the indexed chunks, and the most relevant pieces are returned.
Place a question bubble on the right and highlight two chunk records travelling back as search results.
A left-to-right flow. Start with one long document. Then show it split into several chunks, first without overlap and then with slight overlap. Next show each chunk being indexed for search. End with a user question retrieving two relevant chunks. Read the diagram from left to right to see how document structure affects later retrieval quality.
On hover or tap, highlight how a sentence near a chunk boundary may be lost without overlap but preserved with overlap.
Describe this without the diagram
The diagram explains chunking as a five-stage flow. First there is one long document. Second, the document is split into four smaller chunks. Third, neighbouring chunks share a small repeated region called overlap, which helps preserve context when a sentence sits near a boundary. Fourth, each chunk is indexed separately for later search. Fifth, a user question retrieves the most relevant chunks rather than the entire document. The main message is that chunk size and overlap change what information can be found later.
With reduced motion: A static left-to-right diagram with five labelled panels: long document, split chunks, overlapping chunks, indexed chunk records, and retrieved results for a question. Shared overlap is shown with shading instead of animation.
In search systems that use embeddings and retrieval, long documents are often broken into smaller units before they are indexed. Those units are commonly called chunks. Chunking is the overall process. A chunk is one resulting piece. Chunks is just the plural. Splitting usually refers to the act of cutting the document, but people sometimes use it loosely to mean the whole process.
What goes in is usually raw text such as a report, policy, meeting note or web page. The system then chooses boundaries: for example every 300 words, every few paragraphs, or at section headings. Some methods also keep overlap, meaning a few sentences are repeated into the next chunk so important context is less likely to be cut in half.
What happens next is that each chunk is turned into a numeric representation called an embedding, then stored for later search. When a user asks a question, the system compares the question with those stored chunk embeddings and retrieves the closest matches. The output is usually a small set of chunks that may be shown directly or passed to a language model as source material.
Good chunking improves the chance that retrieval finds a passage with the right meaning and enough context. Poor chunking can hide answers inside pieces that are too large, or strip away meaning by cutting text too aggressively. Chunking is often confused with tokenisation, but tokenisation splits text into very small units such as words or subwords for model processing, while chunking groups text into larger, useful passages for search and retrieval.
How “Checking a shipping manifest against a delivery” maps onto the real thing
| In the story | In the AI system |
|---|---|
| Sticky-tabbed sections of the manifest | Chunks created from a longer document |
| Repeated lines near section breaks | Overlap between chunks to preserve context |
| Looking in the most relevant section first | Retrieval of the most similar chunks |
| 40-page manifest | Original source document |
A worked example
A reporting officer in a regional bureau must prepare a briefing note before close of business, using cleared monitoring reports stored in an approved internal retrieval system.
Starting from: The reports are divided by heading and paragraph, with the final sentence of each chunk repeated in the next. Each chunk retains the report title, section heading and clearance status as metadata.
- During ingestion, the system keeps short paragraphs together and splits longer sections at sentence boundaries rather than in the middle of a sentence.
- Each chunk is converted into an embedding and stored in the search index with its source metadata.
- The officer asks which implementation constraints recur across the cleared reports.
- The query embedding is compared with the chunk embeddings using similarity search. Passages discussing access, staffing and delivery delays rank highly.
- The system returns the highest-ranked passages with their report titles and section headings. The officer checks each passage in the source before drafting.
Result: The officer receives focused passages rather than several complete reports. The overlap preserves qualifications near section boundaries, while the metadata supports source checking and citation in the briefing note.
In a United Nations workplace
Before the annual workplan review, a legal officer supporting a UN Country Team working group searches previous cleared workplans and notes for the file for commitments on data governance. The approved internal retrieval system has divided those documents into sections that can be retrieved separately.
Section-based chunks let the system return the relevant commitment, qualification and follow-up action without retrieving every workplan in full. Sensible overlap also reduces the risk that a condition at the end of one section is separated from the obligation in the next.
Before you try thisDraft workplans and notes for the file may contain personal, partner-sensitive or security-sensitive material. They must not be uploaded to or indexed in a tool that is not approved for that material. Apply the required access controls and retention rules, and verify retrieved text against the source.
Where you may hear it
- During a vendor demo to an ICT governance board
- At a procurement unit's technical evaluation meeting
- In a Headquarters review of a document-search pilot
- During an inter-agency working group review of a retrieval prototype
- At a country office post-pilot meeting
Avoid confusion
Avoid confusion
Often confused with
Tokenisation
The process of breaking text into smaller units a language model can process, such as words, parts of words, punctuation marks or special symbols.
Keyword search
A search method that finds documents by matching query words to indexed text and ranking the matches, often with scoring methods such as BM25.
Semantic similarity
A measure of how close two pieces of text are in meaning, even if they use different words.
Vector database
A specialised system stores numerical meaning-representations of content and quickly finds the closest matches to a query by semantic similarity.
Common misconceptions
Common misconception: Chunking just means cutting text into equal pieces, so any fixed size will do.
Correction: Fixed-size splitting can work, but it is not always enough. If a heading, definition and exception get separated badly, retrieval may miss the meaning or return incomplete evidence. Good chunking often considers document structure, overlap and the intended query type.
Common misconception: Smaller chunks are always better because they are more precise.
Correction: Very small chunks can improve precision, but they can also remove the context needed to interpret a sentence. A rule, date or warning may only make sense together with nearby text. There is usually a trade-off between precision and completeness.
Common misconception: Chunking and tokenisation are the same thing.
Correction: They happen at different scales and for different purposes. Tokenisation breaks text into very small units, often for model input processing. Chunking groups text into larger passages so search and retrieval can return useful sections of a source.
Common misconception: Once chunking is done, retrieval quality mostly depends on the language model.
Correction: Retrieval quality depends heavily on earlier steps such as chunk boundaries, metadata, embeddings, indexing and ranking. A strong language model cannot reliably recover evidence that was never retrieved.
Quick check
Why do many retrieval systems add overlap between neighbouring chunks?
Overlap repeats boundary text so a sentence near a split keeps some context. It does not reduce token counts, preserve metadata, or guarantee that the top result contains the full answer.
Go deeper
Go deeper
The simple version
Take a long document, break it into sensible pieces, and search those pieces instead of the whole document. The goal is to bring back the passage that best answers the question, with enough surrounding context to make sense.
The technical version
Chunking is the segmentation of source documents into retrievable units before indexing. The unit may be based on fixed length, paragraphs, sections, sentence windows, or semantic boundaries. Many pipelines add overlap so content near boundaries appears in adjacent chunks. Each chunk is then embedded and stored with metadata in an index or vector database. At query time, the query embedding is compared against chunk embeddings, often using nearest-neighbour search and cosine or related similarity measures, followed by ranking or reranking. The choice of chunk size, boundary strategy, overlap, metadata and parent-child linking affects recall, precision, latency, storage and how much context can be passed into retrieval-augmented generation.
Reviewed external resources
EncyclopaediaWikipedia search: Chunking (opens in a new tab)Wikipedia
This Wikipedia search results page helps a reader quickly see the range of meanings attached to 'chunking', which is useful for orientation but best for readers who can judge search results carefully.
search results, not one pagelink checked 2026-07-31
CourseEmbeddings (opens in a new tab)Google
This Google course resource introduces embeddings, which suits readers who want the background needed to understand why chunks can be searched by meaning.
Intermediatelink checked 2026-07-31
InteractiveEmbedding Projector (opens in a new tab)TensorFlow
This TensorFlow interactive tool lets readers explore how text or other items can be placed near one another in embedding space, which suits visual learners and technical readers.
Starterlink checked 2026-07-31
VideoVideo explanations: Chunking (opens in a new tab)YouTube search
This YouTube search results page offers multiple video explanations of chunking, which suits readers who prefer spoken and visual teaching but need to choose sources critically.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Chunking (opens in a new tab)Reddit search
This Reddit search results page shows community questions and practical discussions about chunking choices, which suits readers seeking implementation perspectives but not authoritative guidance.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Chunking determines the units a search system can retrieve. Chunks that are too large dilute relevance; chunks that are too small can separate a statement from the context that gives it meaning.