context window
-
Improving RAG Systems with Semantic Firewalls
Read Full Article: Improving RAG Systems with Semantic Firewalls
In the GenAI space, the common approach to building Retrieval-Augmented Generation (RAG) systems involves embedding data, performing a semantic search, and stuffing the context window with top results. This approach often leads to confusion as it fills the model with technically relevant but contextually useless data. A new method called "Scale by Subtraction" proposes using a deterministic Multidimensional Knowledge Graph to filter out noise before the language model processes the data, significantly reducing noise and hallucination risk. By focusing on critical and actionable items, this method enhances the model's efficiency and accuracy, offering a more streamlined approach to RAG systems. This matters because it addresses the inefficiencies in current RAG systems, improving the accuracy and reliability of AI-generated responses.
-
OpenAI’s Quiet Transformative Updates
Read Full Article: OpenAI’s Quiet Transformative Updates
OpenAI has introduced subtle yet significant updates to its models that enhance reasoning capabilities, batch processing, vision understanding, context window usage, and function calling reliability. These improvements, while not headline-grabbing, are transformative for developers building with large language models (LLMs), making AI products 2-3 times cheaper and more reliable. The enhanced reasoning allows for more efficient token usage, reducing costs and improving performance, while the improved batch API offers a 50% cost reduction for non-real-time tasks. Vision accuracy has increased to 94%, making document processing pipelines more accurate and cost-effective. These cumulative advancements are quietly reshaping the AI landscape by focusing on practical engineering improvements rather than flashy new model releases. Why this matters: These updates significantly lower costs and improve reliability for AI applications, making them more accessible and practical for real-world use.
-
Efficient Data Conversion: IKEA Products to CommerceTXT
Read Full Article: Efficient Data Conversion: IKEA Products to CommerceTXT![[Resource] 30k IKEA products converted to text files. Saves 24% tokens. RAG benchmark.](https://www.tweakedgeek.com/wp-content/uploads/2026/01/featured-article-9368-300x171.png)
Converting 30,511 IKEA products from JSON to a markdown-like format called CommerceTXT significantly reduces token usage by 24%, allowing more efficient use of memory for applications like Llama-3. This new format enables over 20% more products to fit within a context window, making it highly efficient for data retrieval and testing, especially in scenarios where context is limited. The structured format organizes data into folders by categories without the clutter of HTML or scripts, making it ready for use with tools like Chroma or Qdrant. This approach highlights the potential benefits of simpler data formats for improving retrieval accuracy and overall efficiency. This matters because optimizing data formats can enhance the performance and efficiency of machine learning models, particularly in resource-constrained environments.
-
ISON: Efficient Data Format for LLMs
Read Full Article: ISON: Efficient Data Format for LLMs
ISON, a new data format designed to replace JSON, reduces token usage by 70%, making it ideal for large language model (LLM) context stuffing. Unlike JSON, which uses numerous brackets, quotes, and colons, ISON employs a more concise and readable structure similar to TSV, allowing LLMs to parse it without additional instructions. This format supports table-like arrays and key-value configurations, enhancing cross-table relationships and eliminating the need for escape characters. Benchmarks show ISON uses fewer tokens and achieves higher accuracy compared to JSON, making it a valuable tool for developers working with LLMs. This matters because it optimizes data handling in AI applications, improving efficiency and performance.