Samsung’s AI Create Talking Avatars Without 3D Modeling

Samsung’s AI Center has developed a model that has the ability to create realistic animated talking heads from images without 3D modeling.

Recently, Scientists and developers from Samsung’s AI Center in Moscow and Skolkovo Institute of Science and Technology created a model that has the ability to generate realistic animated talking heads from images without relying on traditional methods, like 3D modeling.
In a youtube video, Egor Zakharov, said that although it’s possible to create a model through a single image, training it through multiple images "leads to higher realism and better identity preservation."
The new 'Few-shot learning' means the model can start to animate a face using just a few images of a person, or even a single image. For the purpose, a meta training with the VoxCeleb2 data set of videos is carried out before the model can animate previously unseen faces.
According to the paper, Few-Shot Adversarial Learning of Realistic Neural Talking Head Models, during the training process, the model creates three neural networks firstly, the embedded network maps frames to vectors, than a generator network maps facial landmarks in the synthesized video, and finally a discriminator network assesses the realism and pose of the generated images.
Crucially, the system is able to initialize the parameters of both the generator and the discriminator in a person-specific way so that training can be based on just a few images and done quickly, despite the need to tune tens of millions of parameters. We show that such an approach is able to learn highly realistic and personalized talking head models of new people and even portrait paintings,” says the summary of the paper on arXiv.