New Approaches to RAG Models

𝗥𝗔𝗚 𝗵𝗮𝘀 𝗰𝗼𝗺𝗲 𝗮 𝗹𝗼𝗻𝗴 𝘄𝗮𝘆 — 𝗮𝗻𝗱 𝗶𝘁’𝘀 𝗻𝗼 𝗹𝗼𝗻𝗴𝗲𝗿 𝗷𝘂𝘀𝘁 𝗼𝗻𝗲 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲. Today, Retrieval-Augmented Generation (RAG) is a design space. There are many emerging patterns, but in this post, I’m focusing on the 𝘁𝗼𝗽 𝟴 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲𝘀 you should know in 2025. Why? Because the way we retrieve context 𝘥𝘪𝘳𝘦𝘤𝘵𝘭𝘺 𝘴𝘩𝘢𝘱𝘦𝘴 how intelligent, useful, and safe our AI systems are. Here are 8 RAG variants changing how we build with LLMs: • 𝗦𝗶𝗺𝗽𝗹𝗲 𝗥𝗔𝗚 𝘄𝗶𝘁𝗵 𝗺𝗲𝗺𝗼𝗿𝘆 — add past interactions to make responses more grounded • 𝗕𝗿𝗮𝗻𝗰𝗵𝗲𝗱 𝗥𝗔𝗚 — pull from APIs, databases, and knowledge graphs at once • 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗔𝗚 — where an agent decides what to retrieve and when • 𝗛𝘆𝗗𝗲 — generate hypothetical documents to guide more targeted lookups • 𝗦𝗲𝗹𝗳-𝗥𝗔𝗚 — the system rephrases, self-grades, and reflects before generating • 𝗔𝗱𝗮𝗽𝘁𝗶𝘃𝗲 𝗥𝗔𝗚 — chooses the best data source dynamically • 𝗖𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝘃𝗲 𝗥𝗔𝗚 (𝗖𝗥𝗔𝗚) — filters out noisy context using thresholds • 𝗦𝗶𝗺𝗽𝗹𝗲 𝗥𝗔𝗚 — still a valid choice when your data is clean and static    I created this visual guide to help AI engineers, architects, and teams rethink what retrieval can (and should) do. Because retrieval is no longer just “fetch and feed.” It’s evolving into an intelligent layer that brings 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴, 𝗳𝗲𝗲𝗱𝗯𝗮𝗰𝗸, 𝗮𝗱𝗮𝗽𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆, and 𝗰𝗼𝗻𝘁𝗿𝗼𝗹 into GenAI systems. Have I overlooked anything? Please share your thoughts—your insights are priceless to me.

If you're an AI engineer building RAG pipelines, this one’s for you. RAG has evolved from a simple retrieval wrapper into a full-fledged architecture for modular reasoning. But many stacks today are still too brittle, too linear, and too dependent on the LLM to do all the heavy lifting. Here’s what the most advanced systems are doing differently 👇 🔹 Naïve RAG → One-shot retrieval, no ranking or summarization. → Retrieved context is blindly appended to prompts. → Breaks under ambiguity, large corpora, or multi-hop questions. → Works only when the task is simple and the documents are curated. 🔹 Advanced RAG → Adds pre-retrieval modules (query rewriting, routing, expansion) to tighten the search space. → Post-processing includes reranking, summarization, and fusion, reducing token waste and hallucinations. → Often built using DSPy, LangChain Expression Language, or custom prompt compilers. → Far more robust, but still sequential, limited adaptivity. 🔹 Modular RAG → Not a pipeline- a DAG of reasoning operators. → Think: Retrieve, Rerank, Read, Rewrite, Memory, Fusion, Predict, Demonstrate. → Built for interleaved logic, recursion, dynamic routing, and tool invocation. → Powers agentic flows where reasoning is distributed across specialized modules, each tunable and observable. Why this matters now ⁉️ → New LLMs like GPT-4o, Claude 3.5 Sonnet, and Mistral 7B Instruct v2 are fast — so bottlenecks now lie in retrieval logic and context construction. → Cohere, Fireworks, and Together are exposing rerankers and context fusion modules as inference primitives. → LangGraph and DSPy are pushing RAG into graph-based orchestration territory — with memory persistence and policy control. → Open-weight models + modular RAG = scalable, auditable, deeply controllable AI systems. 💡 Here are my 2 cents- for engineers shipping real-world LLM systems: → Upgrade your retriever, not just your model. → Optimize context fusion and memory design before reaching for finetuning. → Treat each retrieval as a decision, not just a static embedding call. → Most teams still rely on prompting to patch weak context. But the frontier of GenAI isn’t prompt hacking, it’s reasoning infrastructure. Modular RAG brings you closer to system-level intelligence, where retrieval, planning, memory, and generation are co-designed. 🛠️ Arvind and I are kicking off a hands-on workshop on RAG This first session is designed for beginner to intermediate practitioners who want to move beyond theory and actually build. Here’s what you’ll learn: → How RAG enhances LLMs with real-time, contextual data → Core concepts: vector DBs, indexing, reranking, fusion → Build a working RAG pipeline using LangChain + Pinecone → Explore no-code/low-code setups and real-world use cases If you're serious about building with LLMs, this is where you start. 📅 Save your seat and join us live: https://lnkd.in/gS_B7_7d

RAG just got smarter. If you’ve been working with Retrieval-Augmented Generation (RAG), you probably know the basic setup: An LLM retrieves documents based on a query and uses them to generate better, grounded responses. But as use cases get more complex, we need more advanced retrieval strategies—and that’s where these four techniques come in: Self-Query Retriever Instead of relying on static prompts, the model creates its own structured query based on metadata. Let’s say a user asks: “What are the reviews with a score greater than 7 that say bad things about the movie?” This technique breaks that down into query + filter logic, letting the model interact directly with structured data (like Chroma DB) using the right filters. Parent Document Retriever Here, retrieval happens in two stages: 1. Identify the most relevant chunks 2. Pull in their parent documents for full context This ensures you don’t lose meaning just because information was split across small segments. Contextual Compression Retriever (Reranker) Sometimes the top retrieved documents are… close, but not quite right. This reranker pulls the top K (say 4) documents, then uses a transformer + reranker (like Cohere) to compress and re-rank the results based on both query and context—keeping only the most relevant bits. Multi-Vector Retrieval Architecture Instead of matching a single vector per document, this method breaks both queries and documents into multiple token-level vectors using models like ColBERT. The retrieval happens across all vectors—giving you higher recall and more precise results for dense, knowledge-rich tasks. These aren’t just fancy tricks. They solve real-world problems like: • “My agent’s answer missed part of the doc.” • “Why is the model returning irrelevant data?” • “How can I ground this LLM more effectively in enterprise knowledge?” As RAG continues to scale, these kinds of techniques are becoming foundational. So if you’re building search-heavy or knowledge-aware AI systems, it’s time to level up beyond basic retrieval. Which of these approaches are you most excited to experiment with? #ai #agents #rag #theravitshow

In the quest to enhance accuracy and factual grounding in AI, the recent RAG-KG-IL framework emerges as a game-changer. This innovative multi-agent hybrid framework is crafted to tackle the persistent challenges of hallucinations and reasoning limitations in Large Language Models (LLMs). Key highlights of the RAG-KG-IL framework: 1) Integrated knowledge architecture: By combining Retrieval-Augmented Generation (RAG) with Knowledge Graphs (KGs), RAG-KG-IL introduces a structured approach to data integration. This method ensures that AI responses are not only coherent but are anchored in verified and structured domain knowledge, reducing the risk of fabrications. 2) Continuous incremental learning: Unlike traditional LLMs requiring retraining for updates, RAG-KG-IL supports dynamic knowledge enhancement. This allows the model to continuously learn and adapt with minimal computational overhead, making real-time updates feasible and efficient. 3) Multi-agent system for reasoning and explainability: The framework employs autonomous agents that enhance both the reasoning process and system transparency. This architecture supports the model's ability to explain its decisions and provide traceable paths from data to conclusions. 4) Empirical validation: In rigorous case studies—including health-related queries from the UK NHS dataset—RAG-KG-IL demonstrated a significant reduction in hallucination rates, outperforming existing models like GPT-4o. The multi-agent framework not only maintained high completeness in responses but also improved reasoning accuracy through structured and contextual understanding. 5) Knowledge graph growth: The framework's ability to dynamically expand its knowledge base is reflected in its enriched relational data. As the system processes more queries, it effectively integrates new knowledge, enhancing its causality reasoning capabilities significantly. #AI #MachineLearning #KnowledgeGraphs #RAG-KG-IL #AIResearch #ontologies #RAG #GraphRAG

Rethinking Knowledge Integration for LLMs: A New Era of Scalable Intelligence Imagine if large language models (LLMs) could dynamically integrate external knowledge—without costly retraining or complex retrieval systems. 👉 Why This Innovation Matters Today’s approaches to enriching LLMs, such as fine-tuning and retrieval-augmented generation (RAG), are weighed down by high costs and growing complexity. In-context learning, while powerful, becomes computationally unsustainable as knowledge scales—ballooning costs quadratically. A new framework is reshaping this landscape, offering a radically efficient alternative to how LLMs access and leverage structured knowledge—at scale, in real time. 👉 What This New Approach Solves Structured Knowledge Encoding: Information is represented as entity-property-value triples (e.g., "Paris → capital → France") and compressed into lightweight key-value vectors. Linear Attention Mechanism: Instead of quadratic attention, a "rectangular attention" mechanism allows language tokens to selectively attend to knowledge vectors, dramatically lowering computational overhead. Dynamic Knowledge Updates: Knowledge bases can be updated or expanded without retraining the model, enabling real-time adaptability. 👉 How It Works Step 1: External data is transformed into independent key-value vector pairs. Step 2: These vectors are injected directly into the LLM’s attention layers, without cross-fact dependencies. Step 3: During inference, the model performs "soft retrieval" by selectively attending to relevant knowledge entries. 👉 Why This Changes the Game Scalability: Processes 10,000+ knowledge triples (≈200K tokens) on a single GPU, surpassing the limits of traditional RAG setups. Transparency: Attention scores reveal precisely which facts inform outputs, reducing the black-box nature of responses. Reliability: Reduces hallucination rates by 20–40% compared to conventional techniques, enhancing trustworthiness. 👉 Why It’s Different This approach avoids external retrievers and the complexity of manual prompt engineering. Tests show comparable accuracy to RAG—with 5x lower latency and 8x lower memory usage. Its ability to scale linearly enables practical real-time applications in fields like healthcare, finance, and regulatory compliance. 👉 What’s Next While early evaluations center on factual question answering, future enhancements aim to tackle complex reasoning, opening pathways for broader enterprise AI applications. Strategic Reflection: If your organization could inject real-time knowledge into AI systems without adding operational complexity—how much faster could you innovate, respond, and lead?

Many companies have started experimenting with simple RAG systems, probably as their first use case, to test the effectiveness of generative AI in extracting knowledge from unstructured data like PDFs, text files, and PowerPoint files. If you've used basic RAG architectures with tools like LlamaIndex or LangChain, you might have already encountered three key problems: 𝟭. 𝗜𝗻𝗮𝗱𝗲𝗾𝘂𝗮𝘁𝗲 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝗠𝗲𝘁𝗿𝗶𝗰𝘀: Existing metrics fail to catch subtle errors like unsupported claims or hallucinations, making it hard to accurately assess and enhance system performance. 𝟮. 𝗗𝗶𝗳𝗳𝗶𝗰𝘂𝗹𝘁𝘆 𝗛𝗮𝗻𝗱𝗹𝗶𝗻𝗴 𝗖𝗼𝗺𝗽𝗹𝗲𝘅 𝗤𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀: Standard RAG methods often struggle to find and combine information from multiple sources effectively, leading to slower responses and less relevant results. 𝟯. 𝗦𝘁𝗿𝘂𝗴𝗴𝗹𝗶𝗻𝗴 𝘁𝗼 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗮𝗻𝗱 𝗖𝗼𝗻𝗻𝗲𝗰𝘁𝗶𝗼𝗻𝘀: Basic RAG approaches often miss the deeper relationships between information pieces, resulting in incomplete or inaccurate answers that don't fully meet user needs. In this post I will introduce three useful papers to address these gaps: 𝟭. 𝗥𝗔𝗚𝗖𝗵𝗲𝗸𝗲𝗿: introduces a new framework for evaluating RAG systems with a focus on fine-grained, claim-level metrics. It proposes a comprehensive set of metrics: claim-level precision, recall, and F1 score to measure the correctness and completeness of responses; claim recall and context precision to evaluate the effectiveness of the retriever; and faithfulness, noise sensitivity, hallucination rate, self-knowledge reliance, and context utilization to diagnose the generator's performance. Consider using these metrics to help identify errors, enhance accuracy, and reduce hallucinations in generated outputs. 𝟮. 𝗘𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁𝗥𝗔𝗚: It uses a labeler and filter mechanism to identify and retain only the most relevant parts of retrieved information, reducing the need for repeated large language model calls. This iterative approach refines search queries efficiently, lowering latency and costs while maintaining high accuracy for complex, multi-hop questions. 𝟯. 𝗚𝗿𝗮𝗽𝗵𝗥𝗔𝗚: By leveraging structured data from knowledge graphs, GraphRAG methods enhance the retrieval process, capturing complex relationships and dependencies between entities that traditional text-based retrieval methods often miss. This approach enables the generation of more precise and context-aware content, making it particularly valuable for applications in domains that require a deep understanding of interconnected data, such as scientific research, legal documentation, and complex question answering. For example, in tasks such as query-focused summarization, GraphRAG demonstrates substantial gains by effectively leveraging graph structures to capture local and global relationships within documents. It's encouraging to see how quickly gaps are identified and improvements are made in the GenAI world.

AI field note: introducing Toolshed from PwC, a novel approach to scaling tool use with AI agents (and winner of best paper/poster at ICAART). LLMs are limited in the number of external tools agents can use at once., usually to about 128 which sounds like a lot, but in a real-world enterprise quickly becomes a limitation. This creates a major bottleneck for real-world applications like database operations or collaborative AI systems that need access to hundreds or thousands of specialized functions. Enter Toolshed, a novel approach from PwC that reimagines tool retrieval and usage that enables AI systems to effectively utilize thousands of tools without fine-tuning or retraining. Toolshed introduces two primary technical components that work together to enable scalable tool use beyond the typical 128-tool limit: 📚 Toolshed Knowledge Bases: Vector databases optimized for tool retrieval that store enhanced representations of each tool, including: tool name and description, argument schema with parameter details, synthetically generated hypothetical questions, key topics and intents the tool addresses Tool-specific metadata for execution. 🧲 Advanced RAG-Tool Fusion: A comprehensive three-phase approach that creatively applies retrieval-augmented generation techniques to the tool selection problem, enhancing tool documents with rich metadata and contextual information accuracy, decomposing queries into independent sub-tasks, and reranking to ensure optimal tool selection. The paper demonstrates significant quantitative improvements over existing methods through rigorous benchmarking and systematic testing: ⚡️ 46-56% improvement in retrieval accuracy (on ToolE and Seal-Tools benchmarks vs. standard methods like BM25). ✨ Optimized top-k selection threshold to systematically balance retrieval accuracy with agent performance and token costs. 💫 Scalability testing: Proven effective when scaling to 4,000 tools. 🎁 Zero fine-tuning required: Works with out-of-the-box embeddings and LLMs. Not too shabby. Toolshed addresses challenges in enterprise AI deployment, offering practical solutions for complex production environments such as cross-domain versatility (we successfully tested across finance, healthcare, and database domains), secure database interactions, multi-agent orchestration, and cost optimization. Congratulations to Elias Lumer, Vamse Kumar Subbiah, and team for winning the best poster award at the International Conference on Agents and AI! For any organization building production AI systems, Toolshed offers a practical path to more capable, reliable tool usage at scale. Really impressive and encouraging work. Link in description.

Researchers from Virginia Tech, Meta, and UC Davis have introduced AR-RAG (Autoregressive Retrieval Augmentation), a novel approach that significantly improves AI image generation by incorporating dynamic patch-level retrieval during the generation process. The Problem with Current Methods: Existing retrieval-augmented image generation methods retrieve entire reference images once at the beginning and use them throughout generation. This static approach often leads to over-copying irrelevant details, stylistic bias, and poor instruction following when prompts contain multiple objects or complex spatial relationships. The AR-RAG Solution Instead of static image-level retrieval, AR-RAG performs dynamic retrieval at each generation step: - Uses already-generated image patches as queries to retrieve similar patch-level visual references - Maintains a database of patch embeddings with spatial context from real-world images - Implements two frameworks: DAiD (training-free) and FAiD (parameter-efficient fine-tuning) - Enables context-aware retrieval that adapts to evolving generation needs Key Results: Testing on three benchmarks (GenEval, DPG-Bench, Midjourney-30K) showed substantial improvements: - 7-point increase in overall GenEval score (0.71 → 0.78) - 2.1-point improvement on DPG-Bench - Significant FID score reduction on Midjourney-30K (14.33 → 6.67) - Particularly strong gains in multi-object generation and spatial positioning tasks Why This Matters: AR-RAG addresses fundamental limitations in current image generation models, especially for complex prompts requiring precise object placement and interaction. The method's ability to selectively incorporate relevant visual elements while avoiding over-copying makes it valuable for applications requiring high fidelity and instruction adherence. The research demonstrates that fine-grained, dynamic retrieval can substantially improve image generation quality while maintaining computational efficiency. AR-RAG: Autoregressive Retrieval Augmentation for Image Generation: https://lnkd.in/g7cjJ32J. Paper and research by Jingyuan Qi, Zhiyang X., Qifan Wang, Huang Lifu

Most people do not look beyond the basic RAG pipeline, and it rarely works out as expected! RAG is known to lack robustness due to the LLM weaknesses, but it doesn't mean we cannot build robust pipelines! Here is how we can improve them. The RAG pipeline, in its simplest form, is composed of a retriever and a generator. The user question is used to retrieve the database data that could be used as context to answer the question better. The retrieved data is used as context in a prompt for an LLM to answer the question. Instead of using the original user question as a query to the database, it is typical to rewrite the question for optimized retrieval. Instead of blindly returning the answer to the user, we better assess the generated answer. That is the idea behind Self-RAG. We can check for hallucinations and relevance to the question. If the model hallucinates, we are going to try again the generation, and if the answer doesn't address the question, we are going to restart the retrieval by rewriting the query. If the answer passes the validation, we can return it to the user. It might be better to provide feedback for the new retrieval and the new generation to be performed in a more educated manner. In the case we have too many iterations, we are going to assume that we just reach a state where the model will apologize for not being able to provide an answer to the question. When we are retrieving the documents, we are likely retrieving irrelevant documents, so it could be a good idea to filter only the relevant ones before providing them to the generator. Once the documents are filtered, it is likely that a lot of the information contained in the documents is irrelevant, so it is also good to extract only what could be useful to answer the question from the documents. This way, the generator will only see relevant information to answer the question. The assumption in typical RAG is that the question will be about the data stored in the database, but this is a very rigid assumption. We can use the idea behind Adaptive-RAG, where we are going to assess the question first and route to a datastore RAG, a websearch or a simple LLM. It is possible that we realize that none of the documents are actually relevant to the question, and we better reroute the question back to the web search. That is part of the idea behind Corrective RAG. If we reach the maximum of web search retries, we can give up and apologize to the user. Here is how I implemented this pipeline with LangGraph: https://lnkd.in/g8AAF7Fw

RAG isn't working as well as you think 🤯 Here's what most people miss: Traditional RAG just matches your query with documents. Like using Ctrl+F with extra steps. But what if your AI model could actually understand the whole context? That's where MemoRAG comes in. Instead of just searching for keywords, It builds a complete understanding of your data. Think of it like this: A regular RAG system is like a student cramming before an exam, Looking up specific answers when needed. MemoRAG is like a student who actually understood the material, Making connections and seeing the bigger picture. Here's what makes it different: 1. Memory-First Approach ↳ It doesn't just search for answers ↳ It builds understanding, very similar to how a human would 2. Smart Retrieval ↳ Generates specific clues from memory ↳ Finds evidence other systems might miss 3. Speed Optimization ↳ 30x faster context pre-filling ↳ Reuses encoded contexts across queries But here's the practical part: You can run it on a single T4 GPU. The lightweight version makes it accessible for smaller teams. Working with RAG systems daily, I've seen their limitations: → Missing context → Shallow understanding → Repetitive processing MemoRAG solves these issues by thinking more like we do. It remembers. It connects. It understands. And the best part?  It's 100% Opensource. Want to try it yourself? Link to the GitHub repo in the comments. P.S. I create AI tutorials and opensource them for free. Your 👍 like and ♻️ repost helps keep me going. Don't forget to follow me Shubham Saboo for daily tips and tutorials on LLMs, RAG and AI Agents.