Occlusion-Aware Video Registration for Highly Non-Rigid Objects

Occlusion-Aware Video Registration for Highly Non-Rigid Objects
Bertram Taetz, Gabriele Bleser, Vladislav Golyanik, Didier Stricker
IEEE Winter Conference on Applications of Computer Vision (WACV-2016), March 7-9, Lake Placid, NY, USA

Abstract:: This paper addresses the problem of video registration for dense non-rigid structure from motion under suboptimal conditions, such as noise, self-occlusions, considerable external occlusions or specularities, i.e. the computation of optical flow between the reference image and each of the subsequent images in a video sequence when the camera observes a highly deformable object. We tackle this challenging task by improving previously proposed variational optimization techniques for multi-frame optical flow (MFOF) through detection, tracking and handling of uncertain flow field estimates. This is based on a novel Bayesian inference approach incorporated into the MFOF. At the same time, computational costs are significantly reduced through iterative pre-computation of the flow fields. As shown through experiments, the resulting method performs superior to other state-of-the-art (MF)OF methods on video sequences showing a highly non-rigidly deforming object with considerable occlusions.