Prof. Dr. Didier Stricker

Dr. Alain Pagani

Keonna Cunningham

Dr. Oliver Wasenmüller

Dr. Gabriele Bleser

Dr. Bruno Mirbach

Dr. Jason Raphael Rambach

Dr. Bertram Taetz

Dr. Muhammad Zeshan Afzal

Mhd Rashed Al Koutayni

Jilliam Maria Diaz Barros

Mahdi Chamseddine

Steve Dias da Cruz

Ahmet Firintepe

Sophie Folawiyo

David Michael Fürst

Kamalveerkaur Garewal

Christiano Couto Gava

Leif Eric Goebel

Tewodros Amberbir Habtegebrial

Jigyasa Singh Katrolia

Andreas Kölsch

Onorina Kovalenko

Muhammad Jameel Nawaz Malik

Nareg Minaskan Karabid

Mohammad Minouei

Manthan Pancholi

Engr. Kumail Raza

Dr. Nadia Robertini

María Alejandra Sánchez Marín

Dr. Kripasindhu Sarkar

Alexander Schäfer

Pascal Schneider

Lukas Stefan Staecker

Dr. Vladislav Golyanik

Dr. Aditya Tewari

André Luiz Brandão

DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR

DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR
Rishav Rishav, Ramy Battrawy, René Schuster, Oliver Wasenmüller, Didier Stricker
International conference in robotics and intelligent systems. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-2020) October 25-29 Las Vegas NV United States Minor Outlying Islands IEEE 10/2020 .

Abstract:: Scene flow is the dense 3D reconstruction of motion and geometry of a scene. Most state-of-the-art methods use a pair of stereo images as input for full scene reconstruction. These methods depend a lot on the quality of the RGB images and perform poorly in regions with reflective objects, shadows, ill-conditioned light environment and so on. LiDAR measurements are much less sensitive to the aforementioned conditions but LiDAR features are in general unsuitable for matching tasks due to their sparse nature. Hence, using both LiDAR and RGB can potentially overcome the individual disadvantages of each sensor by mutual improvement and yield robust features which can improve the matching process. In this paper, we present DeepLiDARFlow, a novel deep learning architecture which fuses high level RGB and LiDAR features at multiple scales in a monocular setup to predict dense scene flow. Its performance is much better in the critical regions where image-only and LiDAR-only methods are inaccurate. We verify our DeepLiDARFlow using the established data sets KITTI and FlyingThings3D and we show strong robustness compared to several state-of-the-art methods which used other input modalities. The code of our paper is available at https://github.com/dfki-av/DeepLiDARFlow.

Back to the overview