NVIDIA is pushing the boundaries of gaming with its latest advancements in artificial intelligence and Unreal Engine 5 integration. The company’s ACE Game Agent SDK and DLSS 4.5 technologies are set to transform the way game characters interact with players, creating more immersive and responsive gaming experiences.
The NVIDIA ACE Game Agent SDK is a lightweight, open-source framework designed for seamless integration into games. It enables developers to create AI companions and NPCs that can understand and respond to player inputs in real time. The SDK’s three core APIs—Agent APIChat APIand RAG API—empower NPCs to perceive, reason, and act within the game world based on real-time player input.
Building on-device AI companions with NVIDIA ACE
One of the most exciting applications of the ACE Game Agent SDK is the creation of on-device AI companions. These companions can understand natural voice interactions, interpret gameplay, and respond dynamically in real time. For example, KRAFTON recently showcased Allyan AI teammate in PUBG: BATTLEGROUNDS that uses natural voice interaction to understand player intent and respond accordingly.
The ACE Game Agent SDK has already been used in real games, such as the experimental in-game AI advisor in Total War: PHARAOH. The advisor, rendered as an ancient Egyptian pharaoh, provides contextual advice to players, using a retrieval-augmented generation (RAG) architecture that queries game data tables to answer player strategy questions in real time.
New NVIDIA ACE Unreal Engine 5 plugins
For developers working with Unreal Engine 5NVIDIA has introduced a new suite of ACE plugins. These plugins support both Blueprint and C++ integration and cover three pillars: automatic speech recognition (ASR)small language models (SLM)and text-to-speech (TTS).
The ASR plugin includes a ready-to-use English language model, with download options for seven additional languages. The SLM plugin provides local GGUF support for low-latency text generation and function calling capabilities. The TTS plugin includes a high-quality Chatterbox Turbo 350M TTS model, example voices, and dedicated plugin content to bring characters to life instantly.
DLSS 4.5 Unreal Engine plugin
The new DLSS 4.5 Unreal Engine plugin adds support for DLSS 4.5 Dynamic Multi Frame Generation, Multi Frame Generation 6X Mode, and the second-resolution transformer for Super Resolution. The plugin provides direct access to the latest DLSS Super Resolution and Frame Generation updates, built on Streamline for consistent integration across DLSS features.
Updated APIs, documentation, and sample code help reduce integration time and make it easier to bring DLSS into both new and existing projects. The plugin offers a consistent integration path across DLSS features, allowing developers to selectively adopt capabilities like Ray Reconstruction or Dynamic Multi Frame Generation.
NVIDIA’s latest advancements in AI technology are set to revolutionize gaming experiences, creating more immersive and responsive interactions between players and game characters. With the introduction of the ACE Game Agent SDK and DLSS 4.5 for Unreal Engine 5, developers have powerful new tools to enhance the gaming experience and push the boundaries of what’s possible in interactive entertainment.
