COGNITO

Augmented and virtual reality are becoming more and more common in systems for user assistance, educational simulators, novel games, and the whole range of applications in between. Technology to automatically capture, recognize, and render human activities is essential for all these applications. The aim of COGNITO is to bring this technology a big step forward.

COGNITO is a European project with activities covering the whole chain from low-level sensor fusion to workflow analysis and assistive visualization. Novel techniques are developed for analyzing, learning, and recording workflows, and then to use the acquired information in the way most suited for the user.

The project emphasizes how the hands are used to interact with objects and tools in the environment. This is an important component needed for making the technology useful in industrial applications. The workflow capturing in COGNITO is built upon the development of a on-body sensor network of miniature inertial and vision sensors. With the sensor network it is possible to accurately track limb motions, and even the fine motor skills of the hands using a wrist mounted camera. This information is then used to identify and classify workflow patterns in the captured movements. In the next step this is used for user monitoring and to develop new interaction paradigms for user-adaptive information presentation.

The focus of the Augmented Vision department is to develop the visual-inertial sensor network and provide the first level of information abstraction from it. This involves developing sensor fusion algorithms to estimate the limb motions and to use the wrist camera to provide detailed hand reconstructions. Augmented Vision also takes part in classifying workflows needed for user monitoring.

Partners

• UNIVERSITY OF BRISTOL
• UNIVERSITY OF LEEDS
• Centre National de la Recherche Scientifiques (CNRS)
• Trivisio Prototyping GmbH
• Center for Computer Graphics (CCG) &
• Technology-Initiative SmartFactory KL .