RAG Is Dead. Who Killed It?

RAG（检索增强生成）这个让AI从文档中找答案的技术，正被一场“内鬼”推翻。这篇文章一针见血地指出：传统RAG的致命伤在于它把文档切片成小段，再用“相似度”来匹配——结果经常抓错段落，导致AI胡编乱造。比如你问“营收下滑”，它可能把“营收增长”和“营收下滑”当成一回事，因为向量空间里它们挨得太近。更糟的是，一个完整的菜谱被切成碎片后，后半段忘了食材名，AI就瞎猜。这根本不是参数调优能解决的，是架构本身出了问题。那怎么解决呢？作者给出了新思路：人类读说明书时不会去算“余弦相似度”，而是翻目录。所以新一代RAG改用图结构或知识图谱，先把文档的章节、交叉引用、表格结构保留下来，再让AI沿着结构找答案。就像先定位到“烤鸡食谱”那章，再读具体步骤——准确率自然飙升。对普通人来说，这意味着以后用AI查合同条款、医疗指南、产品手册时，不会再被凭空捏造的答案误导。无论是公司内部的知识库，还是用户自己问AI“这个药怎么吃”，答案都会更可靠，少了很多“你以为对了其实错了”的坑。

Your RAG pipeline just returned the wrong answer. Again. The query was simple. The doc had the answer. You watched the top-5 chunks come back and none of them held it. The right paragraph sat two pages away from anything your embedding model thought was "similar." So the LLM hallucinated something plausible, your user trusted it, and you spent the afternoon tuning <code>chunk_size</code> from 512 to 1024 and back. This is not a tuning problem. This is the architecture. Vector RAG was a clever hack for a 2022 problem: LLMs had tiny context windows, so we shredded documents into chunks, embedded them, and prayed cosine similarity would find the right shred. It worked well enough on FAQ pages and Notion dumps that we called it a pattern and shipped it into production at every company on earth. Then we pointed it at a 200-page 10-K. A Kubernetes operator manual. A pharma SOP. A legal contract. And it fell apart. <h2> The Chunking Lie </h2> Vector RAG rests on four assumptions. All four break on real documents. Assumption 1: Similarity equals relevance. It doesn't. <code>"revenue declined"</code> and <code>"revenue grew"</code> sit next to each other in embedding space — opposite meanings, near-identical vectors. Domain jargon drifts even worse. <code>"margin"</code> means one thing in finance, another in typography, another in ML. Your general-purpose embedding model doesn't know which doc it's in. Assumption 2: The right answer fits in k chunks. Why is k=5? Because someone's tutorial said so. Real answers span sections, reference earlier definitions, sit inside tables that got shredded on ingest. Top-k is a guess dressed up as a hyperparameter. Assumption 3: Chunk boundaries preserve meaning. They don't. A 512-token window slices mid-sentence, mid-table, mid-recipe. The chunk that contains the answer loses the heading that gives it context. The LLM sees <code>"...bake at 425°F for 40 minutes"</code> and has no idea which dish "it" belongs to. Assumption 4: Documents are bags of text. They aren't. A 10-K has a structure. A manual has chapters. A contract has clauses that reference other clauses. Vector RAG throws all of that away on ingest, then acts surprised when retrieval can't find its way back. Every RAG failure I've debugged in the last two years traces back to one of these four. Bigger embedding models don't fix them. Reranking papers over them. Hybrid BM25 just postpones them. The problem isn't the retriever — it's the premise. <blockquote> Humans don't read a 200-page doc by cosine similarity. They open the table of contents. </blockquote> <h2> How You Actually Find an Answer </h2> Forget databases for a second. Picture a 400-page cookbook, and you want the roast chicken recipe. You don't read all 400 pages looking for the passage most "similar" to chicken. You flip to the contents, find Poultry, turn to that chapter, run your finger down to Roast Chicken, and read the whole recipe — ingredients, oven temp, and timing all together, in order. <a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1dlyvb5rm6jrmamcpgw8.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1dlyvb5rm6jrmamcpgw8.png" alt=" " width="800" height="447"></a> Now imagine doing it the vector-RAG way: tear every page out, shuffle them into a bin, and pull the five loose pages whose words look closest to your question. You might grab the