CES is always filled with the latest and greatest tech, but one announcement stood out for me this year: the continued evolution of NVIDIA’s Avatar Cloud Engine (ACE) technology and its integration with Convai’s AI-driven NPC platform. As someone who has wanted to see gaming evolve and adapt to new technology, I was blown away by the demos and the immense potential showcased, even if I had my concerns.
NVIDIA’s ACE technology combines multiple state-of-the-art AI systems to create stunningly lifelike digital avatars and NPCs. The latest iteration on display at CES 2024 builds on last year’s advancements by incorporating enhanced speech recognition and text-to-speech technology to enable natural conversations between players and computer-controlled characters. Facial animation and lip-syncing are powered by deep learning to match the generated dialogue, creating an incredibly immersive experience.
But it is the collaboration with Convai that makes things really interesting and pushes the concept to the next level. There is a reason the demo at CES got our innovation award, and I am sure we have only seen the tip of the iceberg of its potential. Their platform allows these AI-powered NPCs to dynamically interact with objects and the environment around them based on conversational cues. In one of the demos, an NPC named Jin fluidly pulls out a bottle when the player asks him about his best sake. This level of contextual awareness and autonomy promises to truly transform gameplay and revolutionize how interactive stories and quests are generated.
But even with the excitement swording the tech, there is still a lot of fear this could lead to more layoffs as we move into 2024, but Founder and CEO of Convai, Purnendu Mukherjee, seems to believe the opposite, detailing how it could revolutionize things for the better: “I strongly believe it’s going to create more jobs in those areas, not take away from it. There’s going to be a shift in the kind of job or the exact process of those jobs, and it will require some adapting, but games will need more writers and more voice artists. It will mean more of everything, not less.”
As someone who has been playing games since the 80s—yes, I am that old—the prospect of having unique adventures tailored to my actions and choices is incredibly exciting. No longer will NPCs feel like robotic mannequins that simply stand in place or follow fixed patrol routes. With ACE and Convai’s tech, they can react organically and help pave dynamic stories that unfold in unexpected ways.
Diving into the concept of how the future of games will work, Purnendu Mukherjee believes the process will be more involved with Convai but lead to more immersive worlds to explore. “Now they not only need to do the whole sequence of narrative, but they also need to do 10 times more testing. So it requires more work from the writer, at least for these NPC AI, to get them really engaging”
If that were not enough, Convai can also go beyond dialogue and link directly into how the game is engaged. “You can say anything to the character that you can think of anything, literally anything.” Purnendu Mukherjee explained excitedly, “All while staying in character and staying true to the story will respond to you, acknowledge what you’re saying and respond to you intelligently.” It is the concept many gamers have been dreaming of since we dove head first into our first RPG, and while scary at what this could mean, it is also exciting to see how technology can work in tandem with creatives to make something memorable.
The technology has clearly reached an impressive level of sophistication, although the demos still revealed some of the familiar ‘uncanny valley’ shortcomings of synthetic speech and responses. Memory will also need to be improved to create something truly immersive, but I was assured that the team is actively working on this. But NVIDIA said that major game studios such as Mihoyo, NetEase, Tencent and Ubisoft are already incorporating early versions of ACE into upcoming titles. With more training data and iterations, AI-driven NPC interactions will only get better.
Even the speed at which things are improving is astonishing; in the demo shown off at CES 2024, there were already improvements from previous iterations. “It is effectively infinite of what you can do,” Purnendu Mukherjee detailed when talking about the future of the tech. “What we added recently in this demo is that it’s not just conversation. The conversations will have consequences, meaning these characters will do some actions pertinent to the conversation that you just had. So they may bring you stuff, they may help you, they may be your companion, follow you, or even they may get mad at you and attack you if you say something bad or something.”
CES 2024 may have brought the usual slew of 8K TVs and smart home gadgets, but for me, NVIDIA ACE and its integration with Convai represent the most game-changing innovation. Lifelike AI could fundamentally alter how we experience digital worlds and interact with virtual beings. As an industry veteran, I predict this will be one of the most transformative technologies in years — and the future looks more exciting than ever.
