Morphing billboards for accurate reproduction of shape and shading of articulated objects with an application to real-time hand tracking

Morphing billboards for accurate reproduction of shape and shading of articulated objects with an application to real-time hand tracking
Nils Petersen, Didier Stricker
3rd Conference on Computational Modeling of Objects Presented in Images: Fundamentals, Methods and Applications Conference on Computational Modeling of Objects Presented in Images: Fundamentals, Methods and Applications (compIMAGE-2012), 3rd, September 5-7, Rome, Italy

Abstract:

We present a novel approach to image based rendering of articulated objects that gives reliable estimates of the object’s shape and shading in new, previously unseen poses. The main idea is to combine a kinematic 3D model with a novel extension to billboard rendering in combination with a computationally lightweight pixel-wise morphing technique. The result is both visually appealing and accurate with respect to resolving texture, shading, and elastic deformation of the articulated object. Additionally, we can show experimentally that using our morphing technique allows the formulation of bias-free pixel-wise objective functions with significantly less local optima than using a static texture. Our method is extremely lightweight which makes it very suitable for real-time tracking using a generative model. We demonstrate the applicability of our approach in a hand tracking context on synthetic, ground-truth labeled data and show several examples constructed from synthetic and real hand images.