NVIDIA’s Digital Human Technology Brings AI Characters to Life

Leading AI developers leverage NVIDIA’s technology suite to create lifelike dynamic avatars for applications in gaming, healthcare, finance, and retail.

March 18, 2024 — San Jose, California, USA — 2024 GTC — NVIDIA announced that leading AI application developers are using NVIDIA’s digital human technology to create lifelike avatars for commercial applications and dynamic game characters. The results are being showcased this week at GTC in San Jose, California, as well as in technology demos from companies such as Hippocratic AI, Inworld AI, and UneeQ.

NVIDIA’s Avatar Cloud Engine (ACE) for speech and animation, NVIDIA NeMo™ for language capabilities, and NVIDIA RTX™ for ray-tracing rendering form the foundation for developers to create AI-driven natural language interactions, making conversations more realistic and engaging.

“NVIDIA provides world-class, AI-driven technologies for developers to create digital humans,” said John Spitzer, Vice President of Developer Products and Graphics Performance Technologies at NVIDIA. “These technologies support complex animations and voice dialogues, enhancing the realism of digital interactions.”

World-Class Digital Human Technology

The digital human technology suite includes AI-driven language, speech, animation, and graphics:

● NVIDIA ACE — Empowers developers with facial animation driven by NVIDIA Audio2Face™ and voice communication powered by NVIDIA Riva automatic speech recognition (ASR) and text-to-speech (TTS). ACE microservices are flexible, allowing models to run on the cloud or PC based on local GPU performance to ensure the best user experience.

● NVIDIA NeMo — An end-to-end platform helping developers deliver enterprise-grade generative AI models, including precise data management, advanced model personalization, retrieval-augmented generation, and accelerated performance.

● NVIDIA RTX — A collection of rendering technologies, including RTX Global Illumination (RTXGI) and DLSS 3.5, enabling real-time path tracing in games and applications.

Building Blocks for Digital Humans and Virtual Assistants

To demonstrate new capabilities in digital human technology, NVIDIA collaborated with industry-leading developers Hippocratic AI, Inworld AI, and UneeQ to showcase a range of new technology demos.

Hippocratic AI has created a task-specific, LLM-driven medical agent focused on safety. The agent can call patients, coordinate care tasks, provide preoperative guidance, and manage post-discharge care. At GTC, NVIDIA is extending its solution with Hippocratic AI, leveraging NVIDIA ACE microservices, NVIDIA Audio2Face, NVIDIA Animation Graph, and NVIDIA Omniverse™ Streamer Client to showcase the potential of generative AI medical digital agents.

“Our digital assistants provide helpful, timely, and accurate information to patients worldwide,” said Munjal Shah, Co-Founder and CEO of Hippocratic AI. “NVIDIA ACE technology brings our medical assistant to life with cutting-edge visuals and realistic animations, helping us better communicate with patients.”

UneeQ is a self-service digital human platform creating AI-driven avatars for customer service and engagement applications. Its digital representatives interact with customers in real-time, enhancing purchase confidence. UneeQ integrates the NVIDIA Audio2Face microservice into its platform and combines it with Synanim ML to create highly realistic digital humans, improving customer experience and engagement.

“By combining NVIDIA’s animation AI with our own Synanim ML synthesis animation technology, we enable real-time human-to-human interaction with emotional responsiveness, providing dynamic experiences driven by conversational AI,” said Danny Tomsett, Co-Founder and CEO of UneeQ.

Bringing NPCs to Life in Gaming

NVIDIA ACE is a suite of technologies designed to bring game characters to life. Covert Protocol, a new technology demo developed in collaboration with Inworld AI, pushes the boundaries of character interaction in games. Inworld AI’s engine integrates NVIDIA Riva for accurate speech-to-text conversion and NVIDIA Audio2Face for realistic facial expressions.

Inworld AI’s engine employs a multimodal approach to portray non-player characters (NPCs), integrating cognitive, perceptual, and behavioral systems to present stunning RTX-rendered characters in immersive narratives.

“The combination of NVIDIA ACE microservices with the Inworld engine enables developers to create avatars that drive dynamic storytelling, opening up new possibilities for players to solve puzzles, reason, and play games,” said Kylan Gibbs, CEO of Inworld AI.

Game publishers worldwide are evaluating how NVIDIA ACE can enhance gaming experiences.

Developers across Healthcare, Gaming, Finance, Media & Entertainment, and Retail Leverage ACE

Top game and digital human developers are pioneering the use of ACE and generative AI technologies to transform player interactions with NPCs in games and applications.

Developers and platforms using ACE include Convai, Cyber Agent, Data Monsters, Deloitte, Hippocratic AI, IGOODI, Inworld AI, Media.Monks, MiHoYo, NetEase Games, Perfect World Games, Openstream, Playrix, Quantiphi, Rakuten Securities, Slalom, SoftServe, Tencent, Top Health Tech, Ubisoft, UneeQ, and Unions Avatars.

For more information on NVIDIA ACE, visit https://developer.nvidia.com/ace. Platform developers can integrate the digital human interaction technology suite or individual microservices into their products.

Developers can access AI models in development through an early access program to embark on their NVIDIA ACE journey. To explore available models, developers can now evaluate and access NVIDIA NIM, a set of easy-to-use microservices designed to accelerate Riva and Audio2Face generative AI deployments, on ai.nvidia.com.

About NVIDIA

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s 1999 invention of the GPU sparked the growth of the PC gaming market, redefined modern computer graphics, and accelerated scientific computing. NVIDIA is driving a new era of accelerated computing, transforming industries from gaming to healthcare, autonomous vehicles to robotics. NVIDIA is now a full-stack computing company, with data center-scale solutions transforming every industry. For more information, visit https://nvidianews.nvidia.com/.

###

Media inquiries:

Jade Li

NVIDIA GeForce, Studio PR

Email: jadli@nvidia.com