NVIDIA CUDA
-
Boost GPU Memory with NVIDIA CUDA MPS
Read Full Article: Boost GPU Memory with NVIDIA CUDA MPS
NVIDIA's CUDA Multi-Process Service (MPS) allows developers to enhance GPU memory performance without altering code by enabling the sharing of GPU resources across multiple processes. The introduction of Memory Locality Optimized Partition (MLOPart) devices, derived from GPUs, offers lower latency for applications that do not fully utilize the bandwidth of NVIDIA Blackwell GPUs. MLOPart devices appear as distinct CUDA devices, similar to Multi-Instance GPUs (MIG), and can be enabled or disabled via the MPS controller for A/B testing. This feature is particularly useful for applications where determining whether they are latency-bound or bandwidth-bound is challenging, as it allows developers to optimize performance without rewriting applications. This matters because it provides a way to improve GPU efficiency and performance, crucial for handling demanding applications like large language models.
-
TensorFlow 2.15 Hot-Fix for Linux Installation
Read Full Article: TensorFlow 2.15 Hot-Fix for Linux Installation
A hot-fix has been released for TensorFlow 2.15 to address an installation issue on Linux platforms. The problem arose due to the TensorFlow 2.15.0 Python package requesting unavailable tensorrt-related packages unless pre-installed or additional flags were provided, causing installation errors or downgrades to TensorFlow 2.14. The fix, TensorFlow 2.15.0.post1, removes these dependencies from the tensorflow[and-cuda] installation method, restoring the intended functionality while maintaining support for TensorRT if it is already installed. Users should specify version 2.15.0.post1 or use a fuzzy version specification to ensure they receive the correct version, as the standard version specification will not install the fixed release. This matters because it ensures seamless installation and functionality of TensorFlow 2.15 alongside NVIDIA CUDA, crucial for developers relying on these tools for machine learning projects.