Chunking, Indexing & Retrieval Methods
CoreDesign RAG pipelines: chunking, indexing, and contextual retrieval · Difficulty 3/5
Explanation
RAG grounds Claude in your data by retrieving relevant chunks and placing them in context. The two design choices that matter most are how you split the source material and how you index it for retrieval.
Chunking
Chunking splits source documents into retrievable units. Chunks that are too large dilute relevance and waste tokens; chunks that are too small lose the context needed to answer. There is no universal chunk size — match it to the data shape and query pattern.
Indexing and Retrieval Methods
| Method | How it retrieves | Strength |
|---|---|---|
| Embedding / semantic search | Vector similarity between query and chunk embeddings | Captures meaning and paraphrase |
| Lexical (BM25) | Exact term / keyword matching | Precise on codes, names, rare terms |
| Hybrid (embeddings + BM25) | Combine both, then merge results | Robust across query types |
| Reranking | A reranker reorders the top candidates by relevance | Improves precision of what reaches the model |
The RAG Pipeline
Putting it together, the full pipeline runs: ingest → chunk → (add context) → embed + index (vector + lexical) → retrieve → rerank → assemble context → generate.
Common exam traps
- Treating a bigger context window as a replacement for retrieval. Dumping whole corpora into context causes context rot, higher cost, and worse relevance than targeted retrieval.
- Assuming embedding-only search is always best. Hybrid (embeddings + BM25) plus reranking is the stronger default, especially for exact terms and identifiers that pure semantic search can miss.
Key Takeaways
- Chunk size must match data shape and query pattern — too large dilutes relevance, too small loses context
- Embedding/semantic search captures meaning and paraphrase; BM25/lexical retrieval is precise on exact terms and identifiers
- Hybrid (embeddings + BM25) plus reranking is the stronger default for robustness across query types
- The RAG pipeline order is: ingest, chunk, add context, embed + index, retrieve, rerank, assemble context, generate
- A bigger context window is not a substitute for retrieval — dumping whole corpora into context causes context rot
Glossary Terms
Related Concepts