Paper accepted at CVPR 2020

Our paper with the title “HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map” has been accepted for publication at the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020 (CVPR 2020) which will take place from June 14th to 19th, 2020 in Seattle, Washington, USA. It is the “premier” conference in the field of Computer Vision. Our paper was accepted from 6656 submissions as one of 1470 (acceptance rate: 22 %).

We propose a novel architecture with 3D convolutions for simultaneous 3D hand shape and pose estimation trained in a weakly-supervised manner. The input to our architecture is a 3D voxelized depth map. For shape estimation, our architecture produces two different hand shape representations. The first is the 3D voxelized grid of the shape which is accurate but does not preserve the mesh topology and the number of mesh vertices. The second representation is the 3D hand surface which is less accurate but does not suffer from the limitations of the first representation. To combine the advantages of these two representations, we register the hand surface to the voxelized hand shape. In extensive experiments, the proposed approach improves over the state-of-the-art for hand shape estimation on the SynHand5M dataset by 47.8%. Moreover, our 3D data augmentation on voxelized depth maps allows to further improve the accuracy of 3D hand pose estimation on real datasets. Our method produces visually more reasonable and realistic hand shapes of NYU and BigHand2.2M datasets compared to the existing approaches.

Please find our paper here.

Muhammad Jameel Nawaz Malik, Ibrahim Abdelaziz, Ahmed Elhayek, Soshi Shimada, Sk Aziz Ali, Vladislav Golyanik, Christian Theobalt, Didier Stricker

Please also check out our video on YouTube.

Please contact Didier Stricker for more information.

Three PhDs successfully finished in 2019

We are very happy to announce that three of our PhD students have been able to successfully defend their PhD thesis during 2019!

Mr. Aditya Tewari defended his thesis with the title “Prior-Knowledge Addition to Spatial and Temporal Classification Models with Demonstration on Hand Shape and Gesture Classification” on October 25th in front of the examination commission consisting of Prof. Dr. Didier Stricker (TU Kaiserslautern and DFKI), Prof. Dr. Paul Lukowicz (TU Kaiserslautern and DFKI) and Prof. Dr. Dr. h. c. Dieter Rombach (Fraunhofer IESE, Kaiserslautern).

Mr. Aditya Tewari during his PhD defense on October 25th,  2019

Mr. Vladislav Golyanik defended his thesis with the title „Robust Methods for Dense Monocular Non-Rigid 3D Reconstruction and Alignment of Point Clouds” on November 20th in front of the examination commission consisting of Prof. Dr. Didier Stricker (TU Kaiserslautern and DFKI), Prof. Dr. Antonio Aguado (Universitat Politècnica de Catalunya, Spain) and Prof. Dr. Reinhard Koch (Christian-Albrechts-Universität zu Kiel).

Mr. Vladislav Golyanik during his PhD defense on November 20th, 2019

Mr. Christian Bailer defended his thesis with the title „New Data Based Matching Strategies for Visual Motion Estimation” on November 22nd in front of the examination commission consisting of Prof. Dr. Didier Stricker (TU Kaiserslautern and DFKI), Prof. Dr. Michael Feslberg (Linköpings University, Sweden) and Dr. Margret Keuper (Max-Planck-Institut für Informatik, Saarbrücken).

Mr. Christian Bailer during his PhD defense on November 22nd, 2019

All three PhDs have left our Augmented Vision Department shortly after their defense to pursue a career outside of DFKI.

Two Papers at VISAPP 2020

Our team is presenting two papers at the VISAPP 2020 (15th International Conference on Computer Vision Theory and Applications) conference that is taking place from February 27th – 29th in Valletta, Malta.

The two papers are:

Iterative Color Equalization for Increased Applicability of Structured Light Reconstruction
Torben Fetzer, Gerd Reis, Didier Stricker

Autopose: Large-Scale Automotive Driver Head Pose And Gaze Dataset With Deep Head Pose Baseline
Mohamed Selim, Ahmet Firintepe, Alain Pagani, Didier Stricker

The AutoPOSE dataset can be downloaded from the website at

Paper published in IJCV

The International Journal of Computer Vision (IJCV) is considered one of the top journals in Computer Vision. It details the science and engineering of this rapidly growing field. Regular articles present major technical advances of broad general interest. Survey articles offer critical reviews of the state of the art and/or tutorial presentations of pertinent topics.

We are proud to announce that our paper “SceneFlowFields++: Multi-frame Matching, Visibility Prediction, and Robust Interpolation for Scene Flow Estimation” has been published in the IJCV (for more information click here). It is an extension of our earlier WACV paper “SceneFlowFields: Dense Interpolation of Sparse Scene Flow Correspondences“.

IFA 2019: Intelligent sensor technology for a better posture at the workplace

At the Internationale Funkausstellung (IFA 2019) – the world’s leading trade fair for consumer electronics and home appliances – at the beginning of September in Berlin, researchers from the TU Kaiserslautern (TUK) and the DFKI research area Augmented Reality presented a sensor system that reduces incorrect posture at the workplace. Sensors are used which are attached to different parts of the body such as the arms, spine and legs and which determine the movement sequences. Software evaluates the data and calculates movement parameters such as joint angles on the arm and knee or the degree of flexion or twisting of the spine. The system immediately detects if a movement is carried out incorrectly or if an incorrect posture is adopted. Via a Smartwatch, the system gives the user direct feedback so that he can correct movement or posture. The sensors could be integrated into work clothing and shoes. The run at the IFA was great and the project prototype was assessed very positively throughout. The press was also represented in large numbers.  Please find the links to the articles below.

Video coverage in German:
SWR Landesschau Aktuell Rheinlandpfalz: Video from 06.09.2019, starting 7:45h

Media coverage in German:
Rheinpfalz: Sensoren in der Arbeitskleidung
Ärztezeitung: Digital Health darf bei der IFA nicht fehlen
Elektroniknet: Neue Sensortechnik verspricht bessere Haltung am Arbeitsplatz
Esanum: Haltungsschäden mit Sensortechnik vermeiden Sensoren für eine bessere Körperhaltung
Medica: Bessere Haltung am Arbeitsplatz dank neuer Sensortechnik IFA 2019: Bessere Haltung am Arbeitsplatz dank neuer Sensortechnik IFA 2019: Eine bessere Haltung am Arbeitsplatz dank neuer Sensortechnik IFA 2019: Eine bessere Haltung am Arbeitsplatz dank neuer Sensortechnik IFA 2019: Eine bessere Haltung am Arbeitsplatz dank neuer Sensortechnik
Smarterworld: Sensoren gegen Haltungsschäden Neue Sensortechnik: Bessere Haltung am Arbeitsplatz

Media coverage in English:
Alphagalileo: IFA 2019: Intelligent sensor technology for a better posture at the workplace Sit up straight! Smartwatch that sounds an alarm every time you slump at your desk is being developed by scientists to combat bad posture
Expressdigest: Smartwatch sounds an alarm every time you slump to correct posture
eandt.theiet: Wearables could deter slouching at work, researchers suggest
Elektroniknet: New sensor technology promises better posture at the workplace
France.timesofnews: Smartwatch sounds an alarm every time you slump to correct posture
Longroom: Smartwatch sounds an alarm every time you slump to correct posture IFA 2019: Intelligent sensor technology for a better posture at the workplace IFA 2019: Intelligent sensor technology for a better posture at the workplace
wsbuss: Smartwatch sounds an alarm every time you slump to correct posture
Newsoneplace: Smartwatch sounds an alarm every time you slump to correct posture

Paper at EuroVR Conference

Our team is presenting the paper

Augmented Reality in Physics education: Motion understanding using an Augmented Airtable
Nareg Minaskan Karabid, Jason Raphael Rambach, Alain Pagani, Didier Stricker

at the 16th EuroVR International Conference 2019 in Tallinn, Estonia on October, 23rd-25th.

The paper presents the part of the group’s work within the research project BeGreifen

Two papers at ISMAR 2019

Our team is presenting two papers at the ISMAR 2019 conference in Beijing, China. ISMAR (International Symposium on Mixed and Augmented Reality) is the leading international conference in AR/VR/MR technologies. The two papers on the topics of Object Pose Estimation and SLAM are:

A Shape Completion Component for Monocular Non-Rigid SLAM
Yongzhi Su, Vladislav Golyanik, Nareg Minaskan Karabid, Sk Aziz Ali, Didier Stricker

Deep Multi-State Object Pose Estimation for Augmented Reality Assembly Yongzhi Su, Jason Raphael Rambach, Nareg Minaskan Karabid, Paul Lesur, Alain Pagani, Didier Stricker

Kick-Off meeting of the EU project HyperCOG

Dr. Alain Pagani, and Mohamed Selim (Augmented Vision department) and Ludgar Van Elst (Smart Data and Services department) participated in the Kick-off meeting of the EU project HyperCOG on September 24th, 2019, that was held at IK4-LORTEK in Spain.

HyperCOG project “HYPERCONNECTED ARCHITECTURE FOR HIGH COGNITIVE PRODUCTION PLANTS” addresses the full digital transformation of the process industry and cognitive process production plants through an innovative Industrial Cyber-Physical System (ICPS). It is based on commercially available advanced technologies that will enable the development of a hyper-connected network of digital nodes. The nodes can catch outstanding streams of data in real-time, which together with the high computing capabilities available nowadays, provide sensing, knowledge and cognitive reasoning to the industrial business. The project will strive to demonstrate how data technologies embedded in a CPS platform applied in the process industry can streamline processes, achieve a step gain in efficiency, sustainability and resource utilization and act as a basis for the provision of new services.

In HyperCOG, 14 international partners have joined forces, bringing together the necessary competence in terms of expertise and resources to ensure the achievement of the project goals. In DFKI, the departments “Augmented Vision” and the department “Smart Data and Services” at DFKI Kaiserslautern are involved.

The Augmented Vision department is responsible for developing smart sensing technologies based on visual perception and visual interpretation for different use cases in active industrial plants. Moreover, in industrial plants, with a growing amount of sensors and data, the information load might increase and hinder the monitoring task. Augmented Vision department will carry out the work on AR to present contextualized information in a simple and direct way.

Project partners:
1. SmartFactory Kaiserslautern (GERMANY)
3. Tecnalia Reseach and Innovation (SPAIN)
4. Estia – École Supérieure des Technologies Industrielles Avancées (FRANCE)
5. SIDENOR Aceros Especiales SL (SPAIN)
6. Çimsa Çimento Sanayi Ve Ticaret A.Ş. (TURKEY)
8. MSI – Mondragon Sistemas de Información (SPAIN)
9. U-PEC – Université Paris-Est Créteil (FRANCE)
10. CBS – Cybers Services Zrt (HUNGARY)
11. Ekodenge (TURKEY)
12. 2.0 LCA consultants (DENMARK)
13. Insights Publishers (UNITED KINGDOM).

Contact person: Dr. Alain Pagani, Mohamed Selim

Picture: The HyperCOG partners at the kick-off meeting.

Talk of Dr. Alain Pagani at the HE2Net Symposium 2019

Sylvain Hourlier (THALES) and Alain Pagani (DFKI)

On September 12th, 2019, Dr. Alain Pagani was invited to give a talk on the subject “User monitoring, Human-machine collaboration and Artificial Intelligence” at the 10th HE2Net Symposium.

The HE2Net Symposium, jointly organized by THALES and the Ecole Nationale Supérieure de Cognitique (ENSC) is the annual conference of the Human Engineering and Expertise Network, an association of institutions and companies addressing the topic of human factors in engineering.

The 10th HE2Net Symposium took place in Arcachon, France and was devoted to the topic of Human Factor of Artificial Intelligence. Experts of the Human Factors field and the Artificial Intelligence field could exchange views on questions related to the inclusion of AI methods in human-machine collaboration.

Dr. Alain Pagani presented recent advances on Artificial Intelligence at DFKI, including work on non-invasive human monitoring techniques developed at the Department Augmented Vision.

Photo: Sylvain Hourlier (THALES), co-organizer of the HE2Net Symposium and Alain Pagani (DFKI GmbH).

Papers at IEEE Intelligent Vehicles Conference 2019

The following papers have been presented at the Intelligent Vehicles Symposium (IV) 2019 from 9th to 12th of June in Paris, France:

PWOC-3D: Deep Occlusion-Aware End-to-End Scene Flow Estimation Rohan Saxena, René Schuster, Oliver Wasenmüller, Didier Stricker

DeLiO: Decoupled LiDAR Odometry
Queens Maria Thomas, Oliver Wasenmüller, Didier Stricker

The 2019 IEEE Intelligent Vehicles Symposium (IV’19) is a premier annual technical forum sponsored by the IEEE Intelligent Transportation Systems Society (ITSS). It brings together researchers and practitioners from universities, industry, and government agencies worldwide to share and discuss the latest advances in theory and technology related to intelligent vehicles. Papers concerning all aspects of intelligent vehicles as well as proposals for workshops and specials sessions are invited for IV’19. For more Information click here.