Low-cost, extremely small and energy-efficient: New camera matrix provides precise depth images for automated driving and industrial applications

It is no bigger than a 1-cent piece: a new, ultra-compact and adaptive camera, which, in addition to pictures, also delivers accurate depth information in real-time. As a novel sensor, it offers a wide range of applications in the area of automated driving or manual assembly processes. The system is developed in the project “DAKARA” – design and application of an ultra-compact, energy-efficient and configurable camera matrix for spatial analysis. In the project funded by the German Federal Ministry of Education and Research (BMBF), five partners from industry and research work together.

The camera matrix consists of sixteen square-shaped single cameras, which together function not only as an imaging device, but also as a depth camera. They are arranged on a so-called “wafer”, an approximately 1 millimeter thick structure of polycrystalline semiconductor blanks. The new camera technology comes from AMS Sensors Germany GmbH. With their technology, the camera will not be larger than ten by ten millimeters and about three millimeters thick.

Stephan Voltz, CEO of AMS Sensors Germany, explains the function of the new technology: “The structure as a camera matrix captures the scene from sixteen slightly displaced perspectives in order to calculate the scene geometry (a depth image) using the light field principle. Because such calculations are very high-intensity, an efficient processor is embedded directly into the periphery of the camera matrix to enable real-time applications. ”

The Augmented Vision (AV) research area at the German Research Center for Artificial Intelligence (DFKI) in Kaiserslautern is developing the algorithms for depth image calculations. They are executed in a real-time manner, directly in the camera system. In addition, various applications can be run on the embedded chip for further processing of the generated image data.

Professor Didier Stricker, Head of the Augmented Vision department at DFKI, said: “The depth information provided by the camera system alongside the color information allows a wide range of new applications. The ultra-compact design makes it possible to integrate the new camera into very small filigree components and use it as a non-contact sensor “.

The structure of the camera matrix is reconfigurable. As a result, a more specific layout can be used, depending on the application, for example a different arrangement of the camera matrix in L-shape. The depth image computation can also be adapted to specific requirements for the depth information.

Cameras that provide depth information already exist. However, these send out light to calculate the depth. Disadvantages are the high energy consumption, the large design and high costs. Other passive systems have much lower energy consumption, but are still in the research stage and generally have large shapes and low image rates.

“The DAKARA camera matrix will be the first passive system to provide both color and depth images in real-time, with high image rates, adaptive features, low energy consumption, and a very compact design,” said Oliver Wasenmüller, DFKI Project Manager and co-initiator of the project. The new system is to be used by well-known users from different domains.

Two application scenarios are used to check and demonstrate the developments of DAKARA: A rear-view camera from the partner ADASENS Automotive GmbH is designed to better interpret the rear vehicle environment. This means that finer structures such as curbs or posts can also be detected during automated parking. In addition, the system is designed to recognize people and send warning signals in an emergency. As a result, a tremendous increase in the safety of automated or partially automated driving can be expected. The Bosch Rexroth AG and DFKI (department Innovative Factory Systems) with the Living Lab SmartFactory KL e.V. will be installing a manual assembly process for the workplace assistant. The camera matrix captures objects as well as the hands of the worker by means of the algorithms of the partner CanControls GmbH. The particular challenge is to clearly distinguish objects such as tools or workpieces from the operator’s hands. The depth information of the DAKARA camera is intended to make this separation easier and more precise.

In the next three years the new camera matrix is to be designed, developed and extensively tested in the mentioned scenarios. A first prototype is to be realized by late summer 2018.

The project “DAKARA” is funded by the Federal Ministry of Education and Research (BMBF) within the framework of the “Photonics Research Germany – Digital Optics” program. The project volume totals 3.8 million euros; almost half of which is generated by the industry partners involved.

For more details, please visit the project page: DAKARA

DAKARA Kick-Off at AMS in Nürnberg.
Persons (l.t.r.): Stephan Voltz (AMS), Yuriy Anisimov (DFKI), Benjamin Klimczak (CanControls), Christoph Schiferle (Bosch Rexroth), Alwin Tuschmann (AMS), Oliver Wasenmüller (DFKI), Florian Baumann (ADASENS), Didier Stricker (DFKI), Ariadna Bartra (ADASENS), Martin Geist (Bosch Rexroth), Matthias Möller (Bosch Rexroth), Patrick Bertram (DFKI), Fabian Quint (DFKI)