Deep Multi-State Object Pose Estimation for Augmented Reality Assembly

Deep Multi-State Object Pose Estimation for Augmented Reality Assembly
Yongzhi Su, Jason Raphael Rambach, Nareg Minaskan Karabid, Paul Lesur, Alain Pagani, Didier Stricker
Proceedings of the 18th IEEE ISMAR. IEEE International Symposium on Mixed and Augmented Reality (ISMAR-2019) October 14-18 Beijing China IEEE 2019 .

Abstract:: Neural network machine learning approaches are widely used for object classification or detection problems with significant success. A similar problem with specific constraints and challenges is object state estimation, dealing with objects that consist of several removable or adjustable parts. A system that can detect the current state of such objects from camera images can be of great importance for Augmented Reality(AR) or robotic assembly and maintenance applications. In this work, we present a CNN that is able to detect and regress the pose of an object in multiple states. We then show how the output of this network can be used in an automatically generated AR scenario that provides step-by-step guidance to the user in assembling an object consisting of multiple components.