Agentic retrieval for Amazon Bedrock Managed Knowledge Base

Your users ask multi-part, comparative, and exploratory questions that span PDFs, slides, tickets, transcripts, and web content. Classic single-shot retrieval breaks down on these questions. Answers miss context, support tickets escalate, and analysts waste hours re-running searches. Agentic retrieval for <a href="https://aws.amazon.com/bedrock/knowledge-bases/" target="_blank" rel="noopener">Amazon Bedrock Managed Knowledge Bases</a> is designed for these questions. Consider two questions an analyst might ask: “Compare our 2020 and 2023 strategy. What changed?” and “What are the three biggest risks across product lines?” Neither is a single-shot lookup. A multi-intent question has no single point in embedding space that represents it well, so top-k chunks come back as an average of competing sub-intents. Agentic retrieval plans and iterates over retrieval and can generate a response in the same call. This post focuses on why classic retrieval falls short on multi-part questions, how the AgenticRetrieveStream API works (including request construction and trace parsing), and when to choose it over the standard Retrieve API. <h2 id="why-single-shot-retrieval-falls-short">Why single-shot retrieval falls short</h2> A worked example makes the gap concrete. Suppose you have ingested 25 years of Amazon shareholder letters into a Managed Knowledge Base. Your first question is direct: “What is the most important message in the documents?” The standard <a href="https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_Retrieve.html" target="_blank" rel="noopener">Retrieve API</a> returns five chunks by hybrid score. In our test corpus, the top result is high-scoring but low-value: it covers the Amazon logo color scheme. The following chunks cover hiring, builders, and differentiation. The retriever did its job and ranked by similarity to the query embedding. But “most important message” is vague, and the scorer has no signal for what matters. Now escalate to a realistic question: “Compare how Amazon talked about hiring, long-term investment, and customer obsession in 2020 vs 2023. Where did emphasis shift?” Single-shot retrieval must now serve three intents across two time periods with one query vector. You get one of two outcomes: a scatter of loosely related chunks, or a tight cluster dominated by the strongest signal. Neither is what a human analyst would do. <h3 id="what-a-human-analyst-would-do">What a human analyst would do</h3> A human would decompose the question. They would search hiring in 2020, then hiring in 2023, then investment philosophy in each year. They would read the results, notice gaps, refine the search, and run it again. With enough evidence in hand, they would synthesize. Agentic retrieval automates this workflow as an API. <h3 id="gaps-left-by-single-shot-retrieval">Gaps left by single-shot retrieval</h3> <ul> <li>Multi-part questions: A single embedding may not represent a multi-intent query well, so top-k results tend to blend intents together.</li> <li>Comparative reasoning: “Compare A and B across dimension X” requires targeted retrieval per item, not one blended search.</li> <li>Cross-source questions: When evidence spans multiple knowledge bases, static routing rules are brittle and might not adapt well per query.</li> <li>Sufficiency: Classic retrieval doesn’t judge “enough.” It returns k chunks regardless.</li> <li>Exploration: Some questions need follow-up searches based on what the first results reveal. Single-shot retrieval can’t iterate.</li> </ul> Teams have addressed this gap with custom agent frameworks built on the Retrieve API. They prompt a model to plan and call Retrieve API in a loop, manage sub-queries, deduplicate, and decide when to stop. It works, but it beco