News Archive
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023

Networked, Intelligent and Interactive System for Continuous, Perioperative  Monitoring and Control of an Irrigation Device, as well as for Functional Monitoring of the Lower Urinary Tactonitoring

Networked, Intelligent and Interactive System for Continuous, Perioperative Monitoring and Control of an Irrigation Device, as well as for Functional Monitoring of the Lower Urinary Tactonitoring

Continuous bladder irrigation is the standard after operations on the bladder, prostate, or kidneys to prevent complications caused by blood clots. The irrigation should be monitored constantly, but this is not possible in everyday clinical practice. Therefore, the motivation of VisIMon is to enable automated monitoring. It leads to improved patient care while at the same time relieving the strain on staff.

The aim of the project VisIMon is the development of a small module worn on the body which monitors the irrigation process with the aid of various sensors. The system should seamlessly integrate with the established standard process. Through the cooperation of interdisciplinary partners from industry and research, the necessary sensors are to be developed and combined into an effective monitoring system. Modern communication technology enables the development of completely new concepts how medical devices interact with a hospital. At more than 200,000 applications per year in Germany, the development is extremely attractive not only from a medical but also from an economic point of view.


  • Albert-Ludwigs-Universität Freiburg
  • Fraunhofer Gesellschaft zur Förderung der Angewandten Forschung E.V.
  • Lohmann & Birkner Health Care Consulting GmbH
  • Digital Biomedical Imaging Systems AG


Dr. Dipl.-Inf. Gerd Reis


Sustainable Urban Development Planner for Climate Change Adaptation

The SUDPLAN project aims at developing an easy-to-use web-based planning, prediction, decision support and training tool, for the use in an urban context, based on a what-if scenario execution environment, which will help to assure population’s health, comfort, safety and life quality as well as sustainability of investments in utilities and infrastructures within a changing climate. This tool is based on an innovative and visionary capacity to link, in an ad-hoc fashion, existing environmental simulation models, information and sensor infrastructures, spatial data infrastructures and climatic scenario information in a service-oriented approach, as part of the Single Information Space in Europe for the Environment (SISE). It will provide end users with 3D modeling and simulation as well as cutting edge highly interactive 3D/4D visualization, including visualization on real 3D hardware. The tool includes the SUDPLAN Scenario Management System and three so-called Common Services, which “downscale” regional climate change models using local knowledge, and which will be available for use in all of Europe. Both components will contribute to improved assessment of urban climate change impact. Vital aspects of climate change are considered in 4 carefully selected urban pilot applications located in Austria, the Czech Republic, Germany and Sweden. They cover such diverse applications as: a) extreme rainfall episodes causing problems with uncontrollable, extremely localized runoff, and for drainage and sewage systems, b) hazardous air pollution and high ambient temperature episodes causing health risks, and c) social dynamics (movement of people) as function of climate change and quality of living. DFKI’s role in this European project is the interactive 3D/4D visualization of simulation input and result data on standard 2D as well as on 3D hardware. Furthermore DFKI develops interaction methods for intuitive manipulation and analysis of the aforementioned data.


  • SMHI – Swedish Meteorological and Hydrological Institute, SE
  • AIT – Austrian Institute of Technology GmbH, AT
  • cismet GmbH, DE
  • CENIA – Czech Environmental Information Agency, CZ
  • Apertum IT AB, SE
  • Stockholm Uppsala Air Quality Management Association, SE
  • City of Wuppertal, DE
  • Technische Universität Graz, AT

Virtual Echocardiography System

The objective of the “Virtual Echocardiography Project” is the research and development of innovative techniques and solutions for the achievement of a virtual examination environment for educational purpose in echocardiography.

The visualisation of a beating human heart requires initially the elaboration of an ontological framework for detailed heart-beat descriptions at the medical and at the geometrical level. This ontological framework is important for future virtual tutoring work and also, more generally, for connecting visualisation technology with core Artificial Intelligence technologies at DFKI. In addition to the geometrical heart-beat model it will be of further interest to develop a detailed model based on physiological mechanisms underlying the heart-beat, both for the healthy and the diseased heart. In combination with artifical ultrasound image generation a virtual examination environment will be established.


Design and application of an ultra-compact, energy-efficient and reconfigurable camera matrix for spatial analysis

Design and application of an ultra-compact, energy-efficient and reconfigurable camera matrix for spatial analysis

Within the DAKARA project an ultra-compact, energy-efficient and reconfigurable camera matrix is developed. In addition to standard color images, it provides accurate depth information in real-time, providing the basis for various applications in the automotive industry (autonomous driving), production and many more. The ultra-compact camera matrix is composed of 4×4 single cameras on a wafer and is equipped with a wafer-level optics, resulting in an extremely compact design of approx. 10 x 10 x 3 mm. This is made possible by the innovative camera technology of the AMS Sensors Germany GmbH. The configuration as a camera matrix captures the scene from sixteen slightly displaced perspectives and thus allows the scene geometry (a depth image) to be calculated from these by means of the light field principle. Because such calculations are very high-intensity, close integration of the camera matrix with an efficient, embedded processor is required to enable real-time applications. The depth image calculations, which are researched and developed by DFKI (Department Augmented Vision), can be carried out in real-time in the electronic functional level of the camera system in a manner that is resource-conserving and real-time. Potential applications benefit significantly from the fact that the depth information is made available to them in addition to the color information without further calculations on the user side. Thanks to the ultra-compact design, it is possible to integrate the new camera into very small and / or filigree components and use it as a non-contact sensor. The structure of the camera matrix is reconfigurable so that a more specific layout can be used depending on the application. In addition, the depth image computation can also be reconfigured and thus respond to certain requirements for the depth information.

The innovation of the DAKARA project represents the ultra-ompact, energy-efficient and reconfigurable overall system that provides both color and depth images. Similar systems, which are also found in the product application, are generally active systems that emit light and thus calculate the depth. Major disadvantages of such systems are the high energy consumption, the large design and the high costs. Passive systems have much lower energy consumption, but are still in the research stage and generally have large designs and low image rates. For the first time, DAKARA offers a passive camera, which convinces with an ultra-compact design, high image rates, reconfigurable properties and low energy consumption, leaving the research stage and entering the market with well-known users from different domains.

In order to demonstrate the power and innovative power of the DAKARA concept, the new camera is used in two different application scenarios. These include an intelligent rear-view camera in the automotive field and the workplace assistant in manual production. The planned intelligent rear-view camera of the partner ADASENS Automotive GmbH is capable of interpreting the rear vehicle environment spatially, metrically and semantically compared to currently used systems consisting of ultrasonic sensors and a mono color camera. As a result, even finer structures such as curbsides or poles can be recognized and taken into account during automated parking operations. In addition, the system is able to detect people semantically and to trigger warning signals in the event of an emergency. The DAKARA camera provides a significant contribution to increasing the safety of autonomous or semi-automated driving. A manual assembly process at the Bosch Rexroth AG and DFKI (Department Innovative Factory Systems) is shown in the case of the workplace assistant. The aim is to support and assure the operator of his tasks. For this purpose, the new camera matrix is fixed over the workplace and both objects and hands are detected spatially and in time by the algorithms of the partner CanControls GmbH. A particular challenge is that objects such as tools or workpieces that are held in the hand are very difficult to separate from these. This separation is made possible by the additional provision of depth information by the DAKARA camera. In this scenario, a gripping path analysis, a removal and level control, the interaction with a dialog system and the tool position detection are implemented. The camera is designed to replace a large number of sensors, which are currently being used in various manual production systems by the project partner Bosch Rexroth, thus achieving a new quality and cost level.

In the next three years the new camera matrix will be designed, developed and extensively tested in the mentioned scenarios. A first prototype will be realized by late summer 2018. The project “DAKARA” is funded by the Federal Ministry of Education and Research (BMBF) within the framework of the “Photonics Research Germany – Digital Optics” program. The project volume totals 3.8 million euros; almost half of it is provided by the industry partners involved.


  • AMS Sensors Germany GmbH, Nürnberg (Konsortialführung)
  • Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI), Kaiserslautern (Technische Konsortialführung)
  • ADASENS Automotive GmbH, Lindau
  • Bosch Rexroth AG, Stuttgart
  • CanControls, Aachen

Cognitive Workflow Capturing and Rendering with On-Body Sensor-Networks

Cognitive Workflow Capturing and Rendering with On-Body Sensor-Networks

Augmented and virtual reality are becoming more and more common in systems for user assistance, educational simulators, novel games, and the whole range of applications in between. Technology to automatically capture, recognize, and render human activities is essential for all these applications. The aim of COGNITO is to bring this technology a big step forward.

COGNITO is a European project with activities covering the whole chain from low-level sensor fusion to workflow analysis and assistive visualization. Novel techniques are developed for analyzing, learning, and recording workflows, and then to use the acquired information in the way most suited for the user.

The project emphasizes how the hands are used to interact with objects and tools in the environment. This is an important component needed for making the technology useful in industrial applications. The workflow capturing in COGNITO is built upon the development of a on-body sensor network of miniature inertial and vision sensors. With the sensor network it is possible to accurately track limb motions, and even the fine motor skills of the hands using a wrist mounted camera. This information is then used to identify and classify workflow patterns in the captured movements. In the next step this is used for user monitoring and to develop new interaction paradigms for user-adaptive information presentation.

The focus of the Augmented Vision department is to develop the visual-inertial sensor network and provide the first level of information abstraction from it. This involves developing sensor fusion algorithms to estimate the limb motions and to use the wrist camera to provide detailed hand reconstructions. Augmented Vision also takes part in classifying workflows needed for user monitoring.


  • Centre National de la Recherche Scientifiques (CNRS)
  • Trivisio Prototyping GmbH
  • Center for Computer Graphics (CCG) &
  • Technology-Initiative SmartFactory KL .


Prof. Dr.-Ing. Dipl.-Inf. Gabriele Bleser-Taetz


4-Dimensional Ultrasound

4-Dimensional Ultrasound

The main objective of this project is the improvement of real ultrasound data of the human heart. The spatial representation of the heart in-vivo using ultrasound imaging is currently fairly limited and not of much diagnostic use due to motion artefacts and registration errors in ultrasound-head position tracking. Further on it is technically impossible to scan all medical relevant regions of the heart from a single transducer position.

This is the reason why traditional 2D examinations are done from several positions acquiring the standard slices. Our approach to improve the imaging quality is to merge ultrasound data from different transducer positions intelligently. The motion is recorded by 6-DOF position sensors allowing absolute free positioning of the transducer to get the best beam direction for each region of interest. With specialised techniques in merging ultrasound data from different positions using digital imaging algorithms we are confident to improve the image quality to such an extent that the sensitivity and diagnostic possibilities are significantly enhanced.


  • Klinikum der Bayerischen Julius-Maximilians-Universität Würzburg:

Light and solar management using active and model-predictively controlled components

The research project LiSA is a broad-based joint project in the area of facade, lighting and control technology. The aim is enabling the most energy-efficient operation of office and administrative buildings taking into account user satisfaction. Through exemplary system integration the understanding of the interaction of individual components and the necessity of system solutions are promoted.

At component level technologies are being developed which enable the efficient use of daylight, provide energy-saving lighting with artificial light, and reduce cooling load during summer by means of shading. At the level of sensor technology, a cost-effective sensor is developed, which measures the light conditions as well as heat inputs to the room by solar radiation. A model-predictive control approach optimizes the operation of the components, which can be managed and controlled via wireless communication paths.

With the implementation of the project in a Living Lab Smart Office Space, which is subject to detailed monitoring and in which people use the space according to the actual purpose, it is ensured that the developments are continuously empirically validated and that the results are perceived by users as added value. The people working in the Living Lab have the opportunity to interact with the technology and are thus an essential part of the investigations.


  • Technische Universität Kaiserslautern
  • DFKI GmbH
  • ebök Planung und Entwicklung GmbH
  • Dresden Elektronik Ingenieurtechnik GmbH
  • Agentilo GmbH
  • Herbert Waldmann GmbH & Co. KG


Dr. Dipl.-Inf. Gerd Reis


CAPTURE - 3D-scene reconstruction with high resolution and high dynamic range spherical images

CAPTURE – 3D-scene reconstruction with high resolution and high dynamic range spherical images

Reconstruction of 3D-scenes out of camera images represents an essential technology for many applications, such as 3D-digital-cities, digital cultural heritages, games, tele-cooperation, tactical training or forensic. The objective of the project CAPTURE is to develop a novel approach for 3D scene acquisition and develop corresponding theory and practical methods.

Instead of processing a large amount of standard perspective low resolution video images, we use as input data a few single but full spherical high resolution and high dynamic range (HDR) images. Currently available spherical high resolution cameras are able to record fine texture details and the complete scene from a single point in space. Additionally such cameras provide HDR images yielding consistent color and photometric information. We propose to exploit this new technology focusing on the dense/high-quality 3D reconstruction of both indoor and outdoor environments.

The fundamental issue of the project is to develop novel algorithms that take into account the properties of these images, and thus to push forward the current state of the art in 3D scene acquisition and viewing. In particular we develop novel stable and light-invariant image feature detectors, as well as robust assignment methods for image matching and novel 3D reconstruction/viewing algorithms, which exploit the properties of the images.

The multiple spherical view geometry provides a high amount of redundant information about the underlying environment. This, combined with the consistency of the color and photometric information from HDR images, allows us to develop new methods for robust high-precision image matching and 3D structure estimation, resulting in a high-fidelity textured model of the real scene.

The development of the project CAPTURE makes extensive usage of our Computer Vision Development Framework ARGOS. From the software development side, it is necessary to work with large images and merge information from multiple sources simultaneously. We therefore also put special attention in parallel processing of large amount of data as well as clustering capabilities.

The application of this project is the accurate reconstruction of large scenes which includes industrial facilities, touristic and cultural heritage sites, as well as urban environments.


Dr.-Ing. Alain Pagani


Fusion multimodaler optischer Sensoren zur 3D Bewegungserfassung in dichten, dynamischen Szenen für mobile, autonome Systeme

Fusion multimodaler optischer Sensoren zur 3D Bewegungserfassung in dichten, dynamischen Szenen für mobile, autonome Systeme

Autonomous vehicles will be an indispensable component of future mobility systems. Autonomous vehicles can significantly increase the safety of driving while simultaneously increasing traffic density. Autonomously operating vehicles must be able to continuously and accurately detect their environment and the movements of other road users. To this end, new types of real-time capable sensor systems must be researched. Cameras and laser scanners operate according to different principles and offer different advantages in capturing the environment. The aim of this project is to investigate whether and how the two sensor systems can be combined to reliably detect movements in traffic in real time. The challenge in this case is to suitably combine the heterogeneous data of both systems and to find suitable representations for the geometric and visual features of a traffic scene. These must be optimized to the extent that reliable information can be provided for vehicle control in real time. If such a hybrid sensor system can be designed and successfully built, this could represent a breakthrough for sensor equipment for autonomous vehicles and a decisive step for the implementation of this technology.


Ramy Battrawy, M.Sc.

Dr.-Ing. René Schuster


Consistent dynamic scene reconstruction and property transfer using priors and constraints

Consistent dynamic scene reconstruction and property transfer using priors and constraints

The objective of DYNAMICS is to develop a new methodology for 4D reconstruction of real world scenes with a small number of cameras, as well as to learn statistical models from the captured data sets. A 4D reconstruction refers to a sequence of accurate 3D reconstructions (including geometry, topology and surface properties) of a dynamic (evolving in time) real-world scene. We aim to build a robust lightweight capture system that can be easily installed and used (e.g. in the living room of a house, in outdoor environments, and broadly under various spatial and temporal constraints).

We are developing a novel interactive software system for motion estimation capitalizing on our experience from the predecessor project DENSITY and exploring new directions (new hardware and machine learning methods).

Specifically, the project DYNAMICS can be subdivided into several work packages according to the target scenarios and concerned areas of computer vision:

1) Software for an interactive monocular 4D reconstruction of non-rigid scenes. The main components are modules dealing with non-rigid structure from motion (NRSfM) pipeline and non-rigid registration. Underlying technology will allow to reconstruct non-rigidly deforming scenes with a minimal number of assumptions from a single RGB camera. Target scenarios include endoscopy, capture of facial expressions, small motion and post-factum reconstructions.

2) Software for robust 4D reconstruction from multiple views incorporating optical flow and scene flow with additional assumptions. We plan to assemble a capture studio with five Emergent HT-4000C high-speed cameras (a multi-view setting). Here, we aim at the highest precision and richness of detail in the reconstructions.

3) 3D shape templates with attributes derived from real data using deep learning techniques. The main objective of this work package is to provide statistical models as a prior knowledge in order to increase the robustness and accuracy of reconstructions. Furthermore, the shape templates will allow for more accurate reconstructions of articulated motion (e.g. skeleton poses) from uncalibrated multi-view settings.

DYNAMICS is a BMBF project with an emphasis on development of core technologies applicable in other ongoing and forthcoming projects in the Augmented Vision Lab.