Recent Movies

Augmented reality (AR) presentation enables the creation of natural user interfaces that employ the whole user’s environment as interaction device. Additionally, by using hand based 3D interaction with gestures that have a physical meaning like grabbing, dragging, and dropping this leads to a user experience that is intuitive, since close to the real world’s behavior. We propose a novel approach to an AR-based natural user interface, that goes one step further by enabling the contents of the interface to switch domains from a virtual instance in AR to a physical instance in the real-world. All instances stay associated and changes made to the physical instance will be reflected on the virtual one. Because the behavior of our interface in AR is in key aspects consistent with the real-world, the gap between those domains is made less salient.

We explored a new way to query process and memory data from real objects using Augmented Reality and Natural Interaction. The technology is demonstrated in the context of an industrial application but many diverse applications ranging from training, education, to marketing and entertainment are of course possible.

The basic idea is to capture interaction and user head position/orientation over one single sensor – a camera equipped with a fisheye lens. The video shows early results which consist of the tracking over one single circular marker, and detection of the right and left hand.

In this video, the object's region is described using the P-Channels representation. A reference image with manually marked region is provided by the user. In order to find the position of the screw gun in a new image of the sequence, we exhaustively search the image for the best match with the reference region. This exhautive search is made possible by the use of integral image techniques that we adapted for the P-Channel representation. Note that the video frames are processed independently (no temporal tracking).

Image based interaction represents an attractive way of interacting with the computer. However robustness is the main challenge of this kind of approach. In this video we show a very robust way to detect the hand and derive the pointing direction of the hand. Note the very similar color of the background (the table) and the hand.

Using Circular Markers allow for a detection directly in fisheye cameras. Once the ellipse is detected, the contour is locally wrapped into a perspective view in order to find the camera pose.