NVIDIA's CUDA Multi-Process Service (MPS) allows developers to enhance GPU memory performance without altering code by enabling the sharing of GPU resources across multiple processes. The introduction of Memory Locality Optimized Partition (MLOPart) devices, derived from GPUs, offers lower latency for applications that do not fully utilize the bandwidth of NVIDIA Blackwell GPUs. MLOPart devices appear as distinct CUDA devices, similar to Multi-Instance GPUs (MIG), and can be enabled or disabled via the MPS controller for A/B testing. This feature is particularly useful for applications where determining whether they are latency-bound or bandwidth-bound is challenging, as it allows developers to optimize performance without rewriting applications. This matters because it provides a way to improve GPU efficiency and performance, crucial for handling demanding applications like large language models.
