NPC interactions
-
Unreal Engine Plugin for LLM Gaming
Read Full Article: Unreal Engine Plugin for LLM Gaming
Exploring the integration of local large language models (LLMs) in gaming, a developer has created an Unreal Engine 5 plugin to enhance non-playable character (NPC) interactions. The aim is to move beyond predictable, hard-coded NPC behavior by enabling dynamic dialogue and trait updates through LLMs, while addressing challenges like VRAM limitations and response latency. The project demonstrates that local LLMs can provide creative, contextually appropriate NPC responses, though they are best suited for minor interactions due to potential reliability issues. A technical demo featuring a locally run LLM-controlled NPC highlights the feasibility of this approach, with further optimizations possible through prompt engineering and system configuration. This matters because it showcases a practical application of AI in gaming, enhancing player immersion and interaction with NPCs.
-
WebGPU LLM in Unity for NPC Interactions
Read Full Article: WebGPU LLM in Unity for NPC Interactions
An experiment with in-browser local inference using WebGPU has been integrated into a Unity game, where a large language model (LLM) serves as the NPCs' "brain" to drive decisions at interactive rates. Significant modifications were made to the WGSL kernels to reduce reliance on fp16 and support more operations for forward inference, with unexpected challenges in integrating with Unity due to Emscripten toolchain mismatches. While the WebGPU build offers a performance boost of 3x-10x over CPU depending on hardware, it remains about 10x less efficient than running directly on bare-metal hardware via CUDA. Optimizing WGSL kernels could help bridge this performance gap, and further exploration is needed to understand the limits of WebGPU performance. This matters because it highlights the potential and challenges of using WebGPU for efficient in-browser AI applications, which could revolutionize how interactive web experiences are developed.