Face It!: A Pipeline for Real-Time Performance-Driven Facial Animation

Face It!: A Pipeline for Real-Time Performance-Driven Facial Animation
Jilliam Maria Diaz Barros, Vladislav Golyanik, Kiran Varanasi, Didier Stricker
IEEE (Hrsg.). 25th. IEEE International Conference on Image Processing (ICIP-2019) September 22-25 Taipei Taiwan ISBN 978-1-5386-6249-6 IEEE 2019 .

Abstract:

This paper presents a new lightweight approach for real-time performance-driven facial animation from monocular videos. We transfer facial expressions from 2D images to a 3D virtual character, by estimating the rigid head pose and non-rigid face deformation from detected and tracked 2D facial landmarks. We map the input face into the facial expression space of the 3D head model using blendshape models and formulate a lightweight energy-based optimization problem, which is solved by non-linear least squares at 18 FPS on a single CPU. Our method robustly handles varying head poses and different facial expressions, including moderately asymmetric ones. Compared to related methods, our approach does not require training data, specialised camera setups or graphics cards, and is suitable for embedded systems. We support our claims with several experiments.

Reference:

@inproceedings{diaz2019face,
  title = {Face It!: A Pipeline for Real-Time Performance-Driven Facial Animation},
  author = {D{\'\i}az Barros, Jilliam Mar{\'\i}a and Golyanik, Vladislav and Varanasi, Kiran and Stricker, Didier},
  booktitle = {2019 IEEE International Conference on Image Processing (ICIP)},
  pages = {2209--2213},
  year = {2019},
  organization = {IEEE}
}