CPU performance
-
Qwen3-30B Model Runs on Raspberry Pi in Real Time
Read Full Article: Qwen3-30B Model Runs on Raspberry Pi in Real Time
The ShapeLearn GGUF release introduces the Qwen3-30B-A3B-Instruct-2507 model, which runs efficiently on small hardware like a Raspberry Pi 5 with 16GB RAM, achieving 8.03 tokens per second while maintaining 94.18% of BF16 quality. Instead of focusing solely on reducing model size, the approach optimizes for tokens per second (TPS) without sacrificing output quality, revealing that different quantization formats impact performance differently on CPUs and GPUs. On CPUs, smaller models generally run faster, while on GPUs, performance is influenced by kernel choices, with certain configurations offering optimal results. Feedback and testing from the community are encouraged to further refine evaluation processes and adapt the model for various setups and workloads. This matters because it demonstrates the potential for advanced AI models to run efficiently on consumer-grade hardware, broadening accessibility and application possibilities.
-
30x Real-Time Transcription on CPU with Parakeet
Read Full Article: 30x Real-Time Transcription on CPU with Parakeet
Achieving remarkable speeds in real-time transcription on CPUs, a new setup using NVIDIA Parakeet TDT 0.6B V3 in ONNX format outperforms previous benchmarks, processing one minute of audio in just two seconds on an i7-12700KF. This multilingual model supports 25 languages, including English, Spanish, and French, with impressive accuracy and punctuation capabilities, surpassing Whisper Large V3 in some cases. Users can easily integrate this technology into projects compatible with the OpenAI API, thanks to a developed frontend and API endpoint. This advancement highlights significant progress in CPU-based transcription, offering faster and more efficient solutions for multilingual speech-to-text applications.
-
Qualcomm’s Snapdragon X2 Chips Challenge Intel & AMD
Read Full Article: Qualcomm’s Snapdragon X2 Chips Challenge Intel & AMD
Qualcomm has introduced the Snapdragon X2 Elite and X2 Plus chips, aiming to challenge Intel and AMD with claims of superior performance and efficiency for Windows PCs. The X2 Elite targets high-end laptops, while the X2 Plus is designed for budget machines, both expected to hit the market by the end of the first quarter of 2026. Despite fewer CPU cores, the Plus chips promise competitive performance against low-power Intel chips, and both chipsets boast impressive AI capabilities with an 80 TOPS NPU. While gaming performance might not be groundbreaking, Qualcomm is enhancing graphics driver support, and both chips offer significant power efficiency improvements, potentially leading to multi-day battery life. This development matters as it could shift the competitive landscape in the laptop market, offering consumers more choices and potentially driving innovation and cost-effectiveness.