Common approaches to HCI (Human Computer Interaction) largely consist of stressing the points in which computers exceed human performance. They are based on the assumption that the users working with such a system will have to adapt themselves to their working environment and not vice versa. These approaches do not take sufficiently into account that computers may be very efficient at fast searching and ordering large amounts of data, while humans are much more adept at visually arranging and manipulating data, as well as recognizing relations between different sets of data (meta-data).

Human thinking and knowledge work is heavily dependent on sensing the outside world. One important part of this perception-oriented sensing is the human visual system. It is well-known that our visual knowledge disclosure that is, our ability to think, abstract, remember, and understand visually and our skills to visually organize are extremely powerful. The overall vision of @VISOR is to realize an individually customizable virtual world which inspires the user’s thinking, enables the economical usage of his perceptual power, and adheres to a multiplicity of personal details with respect to his thought process and knowledge work.

The logical conclusion is that by creating a framework that emphasizes the strengths of both humans and machines in an immersive virtual environment, @VISOR can achieve great improvements in the effectiveness of knowledge workers and analysts. The @VISOR project strives to realize this vision by designing methods to present and visualize data in a way that integrates the user into his artificial surroundings seamlessly and gives him/her the opportunity to interact with it in a natural way. In this connection, a holistic context and content-sensitive approach for information retrieval, visualization, and navigation in manipulative virtual environments is introduced. @VISOR addresses this promising and comprehensive vision of efficient man-machine interaction in future manipulative virtual environments by the term “immersion”: a frictionless sequence of operations and a smooth operational flow, integrated with multi-sensory interaction possibilities, which allows an integral interaction of human work activities and machine support. When implemented to perfection, this approach enables a powerful immersion experience: the user has the illusion that he is actually situated in the artificial surroundings, the barrier between human activities and their technical reflection vanishes, and the communication with the artificial environment is seamless and homogeneous. As a result, not only are visually driven thinking, understanding, and organizing promoted, but the identification and recognition of new relations and knowledge is facilitated.

As a matter of course, the study of non-specific, general, real-world information spaces is far too complex to be the aim of @VISOR. Therefore @VISOR will dedicate its studies on virtual environments to personal (virtual) information spaces which are, to a high degree, based on documents i.e., the personal document-based information spaces. In this specific context the above considerations and questions will be concretized and focused.

IMCV project proposes a wearable sensory system, based on inertial motion capture device and visual odometry that can easily be mounted on a robot, as well as on the humans and delivers 3D kinematics in all the environments with an additional 3D reconstruction of the surroundings.

Its objective is to develop this platform for both Exoskeleton and bipedal robot benchmarking.

And it will develop a scenario-generic sensory system for human and bipedal robots and therefore two benchmarking platform will be delivered to be integrated into Eurobench facilities in Spain and Italy for validation tests.

It is planned to use recent advances in inertial measurement units based 3D kinematics estimation that does not use magnetometers and, henceforth, is robust against magnetic interferences induced by the environment or the robot.

This allows a drift-free 3D joint angle estimation of e.g. a lower body configuration or a robotic leg in a body-attached coordinate system.

To map the environment and to correct for possible global heading drift (relative to an external coordinate frame) of the magnetometer-free IMU system, it is planned to fuse the visual odometry stochastically with the IMU system. The recorded 3D point cloud of the stereo camera is used in the post-processing phase to generate the 3D reconstruction of the environment. Therefore a magnetometer-free wearable motion capture system with approximate environment mapping should be created that works for humans and bipedal robots, in any environment, i.e. indoors and outdoors.

To improve localization and measure gait events, a wireless foot pressure insoles will be integrated for measuring ground interaction. Together with the foot insole all the necessary data to reconstruct kinetics and kinematics will be delivered and fully integrated into Robot Operating System (ROS). A user interface will be developed for possible modifications of the skeleton. We also provide validation recordings with a compliant robot leg and with humans, including the computation of key gait-parameters.

Partner

Technische Universität Kaiserslautern, Dekanat Informatik

The joint project ?Software Innovations For the Digital Company? (SINNODIUM) links to the four ongoing projects within the framework of the Cluster Excellence Competition, connects the various fields of research, and guides the overall project to an integrated conclusion. In practice, initial prototype solutions for the next generation of business software??emergent software??are developed that dynamically and flexibly work to make a variety of components from different manufacturers combinable, thus triggering a wave of innovation in digital companies across all sectors.

At SINNODIUM, medium and large software companies therefore work together with research partners on general application scenarios for emergent business software in the areas of Smart Retail (trade), Smart Production (industry), and Smart Services (services and logistics).

Im Rahmen des Projekts ENNOS wird eine kompakte und energieeffiziente Farb- und Tiefenkamera entwickelt, also eine Kamera, die Farbbilder und gleichzeitig 3-dimensionale Informationen zum Abstand von Objekten liefert. Informationen zu Farbe und 3D-Daten werden mittels sogenannter „tiefer neuronaler Netze“ verknüpft, das sind sehr vereinfachte „künstliche Gehirne“: Es wird also „künstliche Intelligenz“ zur rechnergestützten Entscheidungsfindung genutzt.

Ziel ist ein besonders flexibles und leistungsfähiges optisches System, das viele neue Anwendungsmöglichkeiten im Bereich Produktion findet.

Die Auswertung geschieht über Field Programmable Gate Array-Chips (FPGA), das sind programmierbare Integrierte Schaltkreise, die sich an unterschiedliche Aufgaben anpassen lassen. Solche Prozessoren sind besonders flexibel und leistungsfähig, aber von begrenzter Kapazität.

Die Herausforderung liegt darin, die komplexe Struktur und Größe moderner neuronaler Netze effizient in eine passende und kompakte Hardware-Architektur umzuwandeln. Möglich wird dies durch Vorarbeit des Verbundkoordinators Bosch, der eine Vorreiterrolle für solche eingebetteten Lösungen einnimmt.

Unterstützt wird er dabei vom Deutschen Forschungszentrum für Künstliche Intelligenz (DFKI), das sich mit Entscheidungsalgorithmen sowie der Vereinfachung („Pruning“) von neuronalen Netzen beschäftigen wird.

Eine weitere wesentliche Innovation des Projekts ENNOS liegt in der Einführung von ultra-kompakten 3D-Kameras des Projektpartners PMD Technologies AG, der erfolgreich als erster Anbieter eine 3D-Kamera in ein Smartphone integriert hat. Für das Projekt ENNOS werden eine neue Beleuchtungseinheit sowie optische Komponenten für den Industrieeinsatz konzipiert. Dies soll ermöglichen, schwierige Beleuchtungsbedingungen sowie weitere Störeinflüsse aus dem Fertigungsumfeld (z. B. Kalibrierungsungenauigkeiten und Rauschen) zu kompensieren.

Um die große erwartete Leistungsfähigkeit des ENNOS-Konzepts zu demonstrieren, wird die neue (intelligente) Kameraplattform in drei verschiedenen Anwendungsszenarien bei den Verbundpartnern eingesetzt:

Bosch und das DFKI realisieren zusammen die Anwendungen „Ferndiagnose mit automatischer Unkenntlichmachung von Personen“ (Abb. 1a) und „Intelligente Bilderkennung und -analyse mit dem Ziel rein maschinengebundener Produktion“ (Abb. 1b). Die dritte Anwendung „Assistenzsystem für Bestandsaufnahmen“ (Abb. 2) in großen Anlagen wird von den Partnern ioxp GmbH und KSB AG realisiert.

Jedes dieser Szenarien adressiert bestehende Probleme, die durch bisherige Technologien nur bedingt oder gar nicht gelöst werden und daher ein hohes Innovationspotenzial bieten.

Partners

• Robert Bosch GmbH, Gerlingen-Schillerhöhe (Koordinator)
• Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI), Kaiserslautern
• KSB SE & Co. KGaA, Frankenthal
• ioxp GmbH, Mannheim
• pmdtechnologies ag, Siegen (assoziierter Partner)
• ifm eletronic GmbH, Tettnang (assoziierter Partner)

1. Project overview

   The goal of DENSITY is to develop a new methodology for 3D reconstruction suitable to inexperienced end-users. The basic idea relies on the observation that it is difficult to control the user environment and camera settings, and that certain knowledge is needed in order to take “appropriate” pictures. Our easy-to-use, cost-effective scanning solution which is based on such a sensor could make 3D scanning technology more accessible to everyday users. Instead of guiding the user in trying to reduce the number of necessary pictures to a minimum, which leads in practice to very unreliable results, we propose to make use of depth images (RGB-D images) available from using low-cost depth cameras (Kinect or Time-of-Flight sensor). The depth images of those low cost depth cameras are currently of low resolution and noisy, but provide relatively stable results and a good coverage of the perceived area. However, those partial and coarse 3D views still need to be registered and refined in order to be useful in our scanning task.

2. KinectAvatar: Fully Automatic Body Capture Using a Sigle Kinect

   We present a novel scanning system for capturing a full 3D human body model using just a single depth camera and no auxiliary equipment. We claim that data captured from a single Kinect is sufficient to produce a good quality full 3D human model. In this setting, the challenges we face are the sensor’s low resolution with random noise and the subject’s non-rigid movement when capturing the data. To overcome these challenges, we develop an improved super-resolution algorithm that takes color constraints into account. We then align the super-resolved scans using a combination of automatic rigid and nonrigid registration. As the system is of low price and obtains impressive results in several minutes, full 3D human body scanning technology can now become more accessible to everyday users at home. First, we give an overview of our scanning system, easily built as shown in Fig. 1a. The user stands before a Kinect in a range of about 2 meters, such that the full body falls in the Kinect’s range of view. Then, the user simply turns around 360 degrees for about 20 to 30 seconds while maintaining an approximate “T” pose. Fig. 2 shows our scanning results on five users. Our result reproduces the whole human structure well (especially the arms and legs), and can reconstruct detailed geometry such as the face structure and the wrinkles of the clothes. To evaluate the accuracy of the reconstruction, we compare the biometric measurements of the reconstructed human models with data from the actual people in Tab. 1. The values are the average absolute distance among eight people.

We also show average runtime statistics in Tab. 1. The whole processing time for each model is about 14 minutes on average, using an Intel(R) Xeon 2.67GHz CPU with 12GB of memory. Note that 90% of the time in our method is used for computing closest points. Previous work on human body reconstruction can only capture nearly naked human bodies and spends nearly one hour of computation time, and prior work on articulated registration computes the registration frame by frame inK minimization steps, taking nearly two hours to compute.

1. Body Capture Using Multiple Kinects

   In this application, we present a scanning system with muliple Kinects. The Kinects are fixed on the pillars from different views, as Fig. 3a shows. We delete the interference each other of the Kinects by shaking the kinects. And the first scanning result is shown in Fig. 3b. Because the there is only three views, not enough to cover all the human shape, there are some holes on the final mesh. We fill the holes based on a human template . Deform and fit the template human shape to the scanning human, then merge and smooth both mesh together to get a final result.

2. Virtual Clothes Try-on

   There are many applications in virtual reality task with 3d full body model. Online shopping has grown exponentially during the past decade. More andmore customers turn to purchase dresses online. However, the customer can not try out the garment before purchasing, do not know the size of the clothes is suitable or not. With our easy-to-use and low cost Kinect scanning system, the untrained customer can get his own 3D model at home. Then virtual try on clothes with human model, even can interactively edit 2D pattern designs.

Die 3D-Koordinatenmesstechnik dient in der Automobilindustrie zur Beurteilung der Merkmale eines Bauteils oder einer Baugruppe und ist damit eines der wichtigsten Hilfsmittel der Qualitätssicherung. Die effiziente Planung der Messabläufe und deren Integration in den betrieblichen Produktionsablauf sind dabei wichtige Aspekte. In diesem Zusammenhang bietet eine durchgängige Prozesskette, ausgehend von der Messplanung, über die Erzeugung und Simulation des Messprogramms bis hin zur Analyse der Messergebnisse in der Produktion erhebliches Potential, um den Messablauf zu optimieren.

Das Ziel des Projektes iMP – Intelligente Messplanung ist die weitgehende Automatisierung der Messplanung im Rahmen der Qualitätsprüfung von Bauteilen bzw. Baugruppen in der Automobilindustrie. Zu diesem Zweck sollen systemunabhängige Softwaremodule entwickelt werden, die die Erstellung des Messplans weitestgehend automatisieren, so dass der Messingenieur den erzeugten Messplan lediglich überprüfen und ggf. interaktiv überarbeiten muss. Alle Softwaremodule werden in eine einheitliche Benutzeroberfläche integriert, um damit die Voraussetzung für den Einsatz in der produktionsbegleitenden Qualitätsprüfung zu schaffen.

Insgesamt wird durch die automatisierte Messplanung eine Verkürzung der Planungszeiten sowie das Senken von Entwicklungs- und Herstellungskosten erreicht. Anwender der Software sind in erste Linie die Automobilindustrie und deren Zulieferer. Eine Übertragung der Ergebnisse auf andere Industriezweige, wie beispielsweise die Flugzeugindustrie, wird angestrebt.

Tracking multiple persons in 3D with high accuracy and temporal stability in real-time with monocular RGB camera is a challenging task which has a lot of practical applications like 3D human character animation, motion analysis in sports, modeling human body movements and many others. The optical human tracking methods often require usage of multi-view video recordings or depth cameras. Systems which work with monocular RGB cameras are mostly not in real-time, track single person and require additional data like initial human pose to be given. All this implies a lot of practical limitations and is one of the major reasons why optical motion capture systems have not yet seen more widespread use in commercial products. The DFKI research department Augmented Vision presents a novel fully automatic multi-person motion tracking system. The presented system works in real-time with monocular RGB video and tracks multiple people in 3D. It does not require any manual work or a specific human pose to start the tracking process. The system automatically estimates a personalized 3D skeleton and an initial 3D location of each person. The system is tested for tracking multiple persons in outdoor scenes, community videos and low quality videos captured with mobile-phone cameras.

The project is funded by the Federal Ministry of Education and Research (BMBF). Combine tangible, manipulatable objects (“tangibles”) with advanced technologies (“Augmented Reality”) to develop new, intuitive user interfaces. Through interactive experiments, it will be possible to actively support the learning process during the MINT study and to provide the learner with more theoretical information about physics.

In the project interfaces of Smartphones, Smartwatches or Smartglasses are used. For example, a data gadget that allows you to view content through a combination of subtle head movements, eyebrows, and voice commands, and view them on a display attached above the eye. Through this casual information processing, the students are not distracted in the execution of the experiment and can access the objects and manipulate them.

A research project developed as a preliminary study demonstrates the developments. For this purpose, scientists at the DFKI and at the Technical University Kaiserslautern have developed an app that supports students and students in the determination of the relationship between the fill level of a glass and the height of the sound. The gPhysics application captures the amount of water, measures the sound frequency and transfers the results into a diagram. The app can be operated only by gestures of the head and without manual interaction. In gPhysics, the water quantity is recorded with a camera and the value determined is corrected by means of head gestures or voice commands, if required. The microphone of the Google Glass measures the sound frequency. Both information is displayed in a graph that is continuously updated on the display of Google Glass. In this way, the learners can follow the frequency curve in relation to the water level directly when filling the glass. Since the generation of the curve is comparatively fast, the learners have the opportunity to test different hypotheses directly during the interaction process by varying various parameters of the experiment.

In the project, further experiments on the physical basis of mechanics and thermodynamics are constructed. In addition, the consortium develops technologies that enable learners to discuss video and sensor recordings as well as analyze their experiments in a cloud and to exchange ideas with fellow students or to compare results.

Partners

The DFKI is a co-ordinator of five other partners in research and practice: the Technical University of Kaiserslautern, studio klv GmbH & Co. KG Berlin, University of Stuttgart, Con Partners GmbH from Bremen and Embedded Systems Academy GmbH from Barsinghausen.

Funding programm: German BMBF

• Begin: 01.07.2016
• End: 30.06.2019

The virgin stone marble has been used as preferred material for representative buildings and sculptures. Yet, due to its chemical composition and its porosity marble is prone to natural deterioration in outdoor environments, with an accelerating rate since the beginning of industrialization, mainly due to increasing pollution. A basic requirement for a successful restoration and conservation is a regularly repeated assessment of the current object condition and knowledge about prior restoration actions. Ideally the assessment is non-destructive. This requirement is fulfilled for both the optical digitization of objects shape and appearance, and the ultrasound examination used to acquire properties with respect to material quality.

Goal of the joint research project Marmorbild of the University Kaiserslautern, the Fraunhofer Institute (IBMT), and the Georg-August-University Göttingen is the validation of modern ultrasound technologies and digital reconstruction methods with respect to non-destructive testing of facades, constructions and sculptures built from marble. The proof of concept has been provided with prior research.

The planned portable assessment system holds a high potential for innovation. In the future, more objects can be examined cost-effectively in short time periods. Damage can be identified at an early stage allowing for a target-oriented investment of efforts and financial resources.

Partners

Funding by: BMBF

• Funding programm: VIP+
• Grant agreement no.: 03VP00293
• Begin: 01.10.2016
• End: 30.09.2019