The goal of the GreifbAR project is to make mixed reality (MR) worlds, including virtual (VR) and augmented reality (“AR”), tangible and graspable by allowing users to interact with real and virtual objects with their bare hands. Hand accuracy and dexterity is paramount for performing precise tasks in many fields, but the capture of hand-object interaction in current MR systems is woefully inadequate. Current systems rely on hand-held controllers or capture devices that are limited to hand gestures without contact with real objects. GreifbAR solves this limitation by introducing a sensing system that detects both the full hand grip including hand surface and object pose when users interact with real objects or tools. This sensing system will be integrated into a mixed reality training simulator that will be demonstrated in two relevant use cases: industrial assembly and surgical skills training. The usability and applicability as well as the added value for training situations will be thoroughly analysed through user studies.

Partners

NMY – Mixed Reality Communication GmbH Hanauer Landstraße 188, 60314 Frankfurt am Main

Charité – Universitätsmedizin Berlin Charitéplatz 1, 10117 Berlin

Universität Passau vertreten durch den Präsidenten, Prof. Dr. Ulrich Bartosch Innstraße 40, 94032 Passau Wissenschaftliche Leitung: Prof. Dr. Susanne Mayr Lehrstuhl für Psychologie mit Schwerpunkt Mensch – Maschine – Interaktion

Contact

Dr.-Ing. Jason Raphael Rambach

• Jason_Raphael.Rambach@dfki.de
• Phone +49 631 20575 3740

Yongzhi Su, M.Sc.

• Yongzhi.Su@dfki.de
• Phone +49 631 20575 3416

Vision, Identification with Z-sensing Technologies and key Applications

VIZTA project, coordinated by ST Micrelectronics, aims at developing innovative technologies in the field of optical sensors and laser sources for short to long-range 3D-imaging and to demonstrate their value in several key applications including automotive, security, smart buildings, mobile robotics for smart cities, and industry4.0. The key differentiating 12-inch Silicon sensing technologies developed during VIZTA are:

• Innovative SPAD and lock-in pixel for Time of Flight architecture sensors
• Unprecedent and cost-effective NIR and RGB-Z filters on-chip solutions
• complex RGB+Z pixel architectures for multimodal 2D/3D imaging For short-range sensors : advanced VCSEL sources including wafer-level GaAs optics and associated high speed driver.

These developed differentiating technologies allows the development and validation of innovative 3D imaging sensors products with the following highly integrated prototypes demonstrators:

• High resolution (>77 000 points) time-of-flight ranging sensor module with integrated VCSEL, drivers, filters and optics.
• Very High resolution (VGA min) depth camera sensor with integrated filters and optics.

For Medium and Long range sensing, VIZTA also adresses new LiDAR systems with dedicated sources, optics and sensors Technology developments of sensors and emitters are carried out by leading semiconductor product suppliers (ST Microelectronics, Philips, III-V Lab) with the support of equipment suppliers (Amat, Semilab) and CEA Leti RTO.

VIZTA project also include the developement of 6 demonstrators for key applications including automotive, security, smart buildings, mobile robotics for smart cities, and industry4.0 with a good mix of industrial and academic partners (Ibeo, Veoneer, Ficosa, Beamagine, IEE, DFKI, UPC, Idemia, CEA-List, ISD, BCB, IDE, Eurecat). VIZTA consortium brings together 23 partners from 9 countries in Europe: France, Germany, Spain, Greece, Luxembourg, Latvia, Sweden, Hungary, and United Kingdom.

Partners :

Universidad Politecnica Catalunya
Commisariat a l Energie Atomique et aux Energies Alternatives (CEA Paris)
Fundacio Eurecat
STMICROELECTRONICS SA
BCB Informática y Control
Alter Technology TÜV Nord SA
FICOMIRRORS SA
Philips Photonics GmbH
Applied Materials France SARL
SEMILAB FELVEZETO FIZIKAI LABORATORIUM RESZVENYTARSASAG
ELEKTRONIKAS UN DATORZINATNU INSTITUTS
LUMIBIRD
IEE S.A.
IBEO Automotive Systems GmbH
STMICROELECTRONICS RESEARCH & DEVELOPMENT LTD
STMICROELECTRONICS SA
IDEMIA IDENITY & SECURITY FRANCE
Beamagine S.L.
Integrated Systems Development S.A.
VEONEER SWEDEN AB
III-V Lab
STMICROELECTRONICS (ALPS) SAS
STMICROELECTRONICS GRENOBLE 2 SAS