Nvidia wants to use the power of generative AI to allow developers to create NPCs capable of recognizing natural language queries and answering them on the spot during games. In its Computex 2023 event in Taipei, Nvidia CEO Jensen Huan shared a real-time game demo render of an NPC responding to a player who interacted in natural language.
The company said it is possible through the NVIDIA Avatar Cloud Engine (ACE) for Games, a custom AI model foundry service that will arm NPCs with AI-powered natural language. According to Nvidia, ACE includes customizable large language model NeMo (offering foundation language models and model customization tools), Riva (allowing automatic speech recognition and text-to-speech capabilities), and Omniverse Audio2Face (for expressive facial animation creation using audio source).
In the clip shared, a scene between the player and an NPC in a ramen shop was rendered in collaboration with startup Convai. According to Nvidia, the modules included in ACE were integrated into the Convai services platform, and it also used Unreal Engine 5 and MetaHuman to produce the rendered scene. Aside from those, the NVIDIA Lightspeed Studios art team also used NVIDIA RTX Direct Illumination and DLSS.
Some might still find the scene far from perfect, but the idea of being able to interact with a game NPC using natural language opens a whole new possibility in the gaming industry. Using this new creation, developers will be able to make their games more interactive. Players, meanwhile, should soon find it easier to gather game details through casual interactions with the AI within games.
According to the company, the modules of ACE are being separately used on different games now, including Audio2Face in S.T.A.L.K.E.R. 2 Heart of Chornobyl and Fort Solis. Nvidia’s Jason Paul also told The Verge that Nvidia’s tech could cover more than just one character at a time and can even be used to create a conversation between NPCs.
