Improving RAG Systems with Semantic Firewalls

In the GenAI space, the common approach to building Retrieval-Augmented Generation (RAG) systems involves embedding data, performing a semantic search, and stuffing the context window with top results. This approach often leads to confusion as it fills the model with technically relevant but contextually useless data. A new method called “Scale by Subtraction” proposes using a deterministic Multidimensional Knowledge Graph to filter out noise before the language model processes the data, significantly reducing noise and hallucination risk. By focusing on critical and actionable items, this method enhances the model’s efficiency and accuracy, offering a more streamlined approach to RAG systems. This matters because it addresses the inefficiencies in current RAG systems, improving the accuracy and reliability of AI-generated responses.

In the rapidly evolving field of Generative AI (GenAI), a common practice is to rely on Retrieval-Augmented Generation (RAG) systems that use a simplistic approach: Semantic Search followed by selecting the Top K results and stuffing the context window. This method assumes that the language model can sift through the noise and extract meaningful insights. However, this approach often leads to overloading the model with vast amounts of technically relevant but contextually irrelevant data, which can confuse the model rather than enhance its capabilities. The challenge lies in the fact that language models are not inherently equipped to filter out noise effectively when execution is involved, leading to inefficiencies and potential inaccuracies.

To address this issue, a novel concept called “Scale by Subtraction” has been proposed. This approach involves using a deterministic Multidimensional Knowledge Graph to pre-filter information before it reaches the language model. By employing specific dimensions such as Identity, Organizational Hierarchy, and Service Ownership, the graph slices through the noise, presenting only the most relevant subset of data to the model. This method not only reduces the volume of data the model needs to process but also enhances the quality of the information, thereby improving the model’s performance and reducing the risk of hallucinations—where the model generates inaccurate or nonsensical outputs.

Empirical results from internal prototyping demonstrate the effectiveness of this approach. Traditional RAG systems retrieved a larger number of items, many of which were irrelevant or outdated, while the graph-filtered method retrieved fewer items that were all critical and actionable. The noise reduction achieved was approximately 99%, and the risk of hallucination was significantly minimized. This highlights the potential of integrating deterministic graphs into the GenAI workflow, not as a means to answer questions directly, but to eliminate incorrect answers and streamline the model’s decision-making process.

The implications of this approach are significant for the future of AI systems. By refining the input data through deterministic filtering, AI models can operate more efficiently and deliver more accurate results. This method challenges the current paradigm of treating the context window as a catch-all for information and encourages a more disciplined, strategic approach to data handling in AI systems. As the field continues to evolve, adopting such innovative strategies could lead to more robust and reliable AI applications, ultimately enhancing their utility across various domains. The conversation around layering deterministic graphs before vector searches is an exciting development that could reshape how we think about and implement AI technologies.

Read the original article here