AI & Technology Updates

  • Cogitator: Open-Source AI Runtime in TypeScript


    I (almost) built an open-source, self-hosted runtime for AI agents in TypeScript...Cogitator is an open-source, self-hosted runtime designed to orchestrate AI agents and LLM swarms, built with TypeScript to offer type safety and seamless web integration. It provides a universal LLM interface that supports multiple AI platforms like Ollama, vLLM, OpenAI, Anthropic, and Google through a single API. The system is equipped with a DAG-based workflow engine, multi-agent swarm strategies, and sandboxed execution using Docker/WASM for secure operations. With a focus on production readiness, it utilizes Redis and Postgres for memory management and offers full observability features like OpenTelemetry and cost tracking. This matters because it aims to provide a more stable and efficient alternative to existing AI infrastructures with significantly fewer dependencies.


  • EdgeVec v0.7.0: Browser-Based Vector Search


    EdgeVec v0.7.0: Run Vector Search in Your Browser — 32x Memory Reduction + SIMD AccelerationEdgeVec v0.7.0 is a browser-based vector database designed to provide local AI applications with cloud-like vector search capabilities without network dependency. It introduces significant updates such as binary quantization for a 32x memory reduction, SIMD acceleration for up to 8.75x faster processing, and IndexedDB persistence for data retention across sessions. These features enable efficient local document search, offline retrieval-augmented generation (RAG), and privacy-preserving AI assistants by allowing data to remain entirely on the user's device. This matters because it empowers users to perform advanced searches and AI tasks locally, maintaining privacy and reducing reliance on cloud services.


  • EdgeVec v0.7.0: Fast Browser-Native Vector Database


    [P] EdgeVec v0.7.0: Browser-Native Vector Database with 8.75x Faster Hamming Distance via SIMDEdgeVec is an open-source vector database designed to run entirely in the browser using WebAssembly, offering significant performance improvements in its latest version, v0.7.0. The update includes an 8.75x speedup in Hamming distance calculations through SIMD optimizations, a 32x memory reduction via binary quantization, and a 3.2x acceleration in Euclidean distance computations. EdgeVec enables browser-based applications to perform semantic searches and retrieval-augmented generation without server dependencies, ensuring privacy, reducing latency, and eliminating hosting costs. These advancements make it feasible to handle large vector indices in-browser, supporting offline-first AI tools and enhancing user experience in web applications. Why this matters: EdgeVec's advancements in browser-native vector databases enhance privacy, reduce latency, and lower costs, making sophisticated AI applications more accessible and efficient for developers and users alike.


  • LLMs Play Mafia: Great Liars, Poor Detectives


    A developer has created a platform where large language models (LLMs) engage in games of Mafia against each other, revealing intriguing insights into their capabilities. While these AI models excel at deception, often proving to be adept liars, they struggle significantly with the detective aspect of the game, indicating a gap in their ability to deduce and analyze information effectively. This experiment highlights the strengths and limitations of LLMs in social deduction games, shedding light on their potential and areas for improvement in understanding and reasoning tasks. Understanding these capabilities is crucial for developing more nuanced and effective AI systems in the future.


  • TOPAS-DSPL: Dual-Stream Transformer for Reasoning


    [P] TOPAS-DSPL: A 15M param Dual-Stream Recursive Transformer achieving 24% on ARC-2TOPAS-DSPL is a neuro-symbolic model that utilizes a dual-stream recursive transformer architecture to enhance small-scale reasoning tasks. By employing a "Bicameral" latent space, it separates algorithmic planning from execution state, which reduces "Compositional Drift" compared to traditional monolithic models. With a parameter count of approximately 15 million, it achieves a 24% accuracy on the ARC-AGI-2 Evaluation Set, showing a significant improvement over standard Tiny Recursive Models. The model's architecture addresses the "forgetting" problem in recursive loops by decoupling rule generation from state updates, and the open-sourcing of its training pipeline allows for independent verification and further development. This matters as it demonstrates significant advancements in reasoning models, making them more accessible and effective for complex problem-solving tasks.