Cyberpunk 2077's dialogue was lip-synced by AI

Cyberpunk 2077 is just a few weeks away and the action-packed RPG will launch with

and subtitle options for several others. CD Projekt Red is aiming to add a deeper layer of immersion for players by using artificial intelligence to in multiple languages.

The game’s lead character technical director, Mateusz Popławski, said in a presentation (via

on Twitter) that CDPR targeted better lip sync quality than in The Witcher 3. It wanted to do so in 10 languages: English, German, Spanish, French, Italian, Polish, Brazilian Portuguese, Russian, Mandarin and Japanese.

The goal was to do all of that for every character in the open world. Because of the game’s enormous scope, CDPR needed to do all of that with zero facial motion capture. To do so, the studio tapped into Jali Research’s lip syncing and facial animation tech to procedurally generate how the characters’ faces move.

Some of the characters in Cyberpunk 2077 speak multiple languages, sometimes switching between them in the same sentence. To account for that, transcript tagging was used. The tags also helped to adjust a character’s facial expressions when their emotional state changed within a line of dialogue. The system also used audio analysis to replicate the emotions of a vocal performance in the animations.

CDPR used algorithms to help it animate the cutscenes in The Witcher 3, and this is an evolution of that approach. Many games use motion capture to lip sync dialogue in just one language, which can break the immersion a bit for players who switch to a different one.

While the procedurally generated animations in Cyberpunk 2077 might not be quite as detailed or as expressive as those in some other AAA games, they could add to the experience for many players who’d prefer to play in different languages. You’ll be able to take a closer look at how that works in practice when the game arrives on November 19th.

sonos sonos One (Gen 2) - Voice Controlled Smart Speaker with Amazon Alexa Built-in - Black read more
In this article: accessibility, langauges, cyberpunk 2077, cyberpunk2077, jali research, jaliresearch, procedural generation, proceduralgeneration, ai, artificial intelligence, artificialintelligence, news, gaming

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.

Comments

Share

4 Shares

sonos sonos One (Gen 2) - Voice Controlled Smart Speaker with Amazon Alexa Built-in - Black read more

all right reserved for yahoo news

NEXT The Roomba i7+ robot vacuum returns to its all-time low price