Matching Retrieval Strategy to Data Shape & Query Pattern
CoreMatch retrieval strategy to data shape and query pattern · Difficulty 3/5
Explanation
There is no single right retrieval design. The architect matches strategy to the data and the queries rather than reaching for a favorite technique on every project.
Strategy by Data Shape
- Structured / tabular data with precise lookups → query the source of truth directly (SQL/API) rather than embedding it. Embedding a database is rarely the right move when the database can just be queried.
- Exact identifiers, codes, names → lexical/BM25 shines; pure semantic search may miss exact matches on rare terms.
- Natural-language, paraphrase-heavy questions → semantic embeddings, with reranking added for precision.
- Mixed corpora → hybrid retrieval, combining embeddings and BM25.
- Small, stable reference set that fits comfortably → it may be cheaper and simpler to place it in a cached prompt prefix than to build a full retrieval pipeline for it.
The Core Principle
The point is *fit*: retrieval strategy follows from what the data looks like and how users ask, not from a preferred technique applied uniformly. A production integration will often mix several of these strategies side by side — direct queries for structured data, hybrid retrieval for a mixed document corpus, and a cached prefix for a small stable glossary — rather than forcing everything through one pipeline.
Key Takeaways
- Structured/tabular precise lookups should query the source of truth directly, not be embedded
- Exact identifiers, codes, and names favor lexical/BM25; semantic search may miss them
- Natural-language, paraphrase-heavy questions favor semantic embeddings plus reranking
- Mixed corpora call for hybrid retrieval combining embeddings and BM25
- A small, stable reference set may be cheaper placed in a cached prompt prefix than built into a retrieval pipeline
Glossary Terms
A Claude API feature that caches frequently-used prompt content (system prompts, large documents, tool definitions) to reduce cost and latency on repeated API calls. Cached tokens are billed at a discounted rate. Cache has a TTL that resets on each use. Must be enabled by marking content with cache_control.
A pattern that dynamically retrieves relevant information from an external knowledge base and injects it into the context window based on the current query. Allows Claude to reason over large document sets without fitting everything in context at once.
Related Concepts