Jobs

The Augmented Vision Department of DFKI led by Prof. Dr. Didier Stricker is offering a Student
Assistant position for curious and passionate students, who want to explore the field of explainable
autonomous driving and human-like perception (More guidance and specifics about the topic will be discussed in
the initial interview)

Main Tasks:
• Work with the driving simulator at DFKI and assist in data collection.
• Design driving scenarios for driving according to specific use cases.
• Write scripts in Python to process and extract sensor data from a simulated vehicle.
• Develop machine learning models to simulate human-like perception in autonomous vehicles.

Requirements
• Good knowledge of concepts in Computer Vision, Deep Learning and LLMs.
• High-level experience in Python and PyTorch/TensorFlow
• Basic driving experience (optional but preferred).
• Interest in hands-on work with driving simulator and wearable sensors.

References
• Marcu, A.-M. LingoQA: Visual Question Answering for Autonomous Driving (arXiv:2312.14115). arXiv.
https://doi.org/10.48550/arXiv.2312.14115
• Kotseruba, I., & Tsotsos, J. K. (2024). SCOUT+: Towards Practical Task-Driven Drivers’ Gaze Prediction
(arXiv:2404.08756). arXiv. http://arxiv.org/abs/2404.08756
• Alletto, S., Palazzi, A., Solera, F., Calderara, S., & Cucchiara, R. (2016). DR(eye)VE: A Dataset for Attention-Based Tasks
with Applications to Autonomous and Assisted Driving. 2016 IEEE Conference on Computer Vision and Pattern Recognition
Workshops (CVPRW), 54–60. https://doi.org/10.1109/CVPRW.2016.14

• Apply latest by: 07.11.2024

Contact:
Shreedhar Govil, Researcher
Augmented Vision, DFKI
Trippstadter Straße 122, 67663 Kaiserslautern
Email: shreedhar.govil@dfki.de

GANs have made significant strides since their inception in 2014, demonstrating remarkable capabilities in generating realistic audio and video mixes, as well as complex geometries. However, a persistent challenge lies in ensuring the accuracy of the generated results. While the visual appeal of these results may be convincing, they often fail to faithfully represent the genuine geometric properties they aim to emulate. A pertinent example is the simulation of paper folding and crumpling, where maintaining the inherent geometric characteristics of the sheet (e.g., preventing stretching) is crucial. Although deterministic simulations of paper deformation have been developed, comprehending and replicating them often necessitate an extensive understanding of material physics, mathematics, and computer graphics. One potential approach to address this challenge involves harnessing the power of GANs or other network architectures, such as variational autoencoders, to analyze 3D geometries and generate highly accurate representations. However, several hurdles must be overcome, including operating effectively within the 3D space and establishing a robust methodology to evaluate the fidelity of the output geometries. An intriguing application of such methods for generating 3D geometries lies in their ability to rapidly generate synthetic data with exceptional accuracy. This synthesized data can then be employed for training purposes in various domains, offering a valuable resource for enhancing learning algorithms and expanding their applicability.

Tasks

State-of-the-art review on GANs and paper simulation
Generating paper geometries using simulation
Use GANs to generate geometries and designing accuracy and comparison methodology
Evaluating generated results from GANs and possible refinement strategies

Requirements

3D computer vision, Computer graphics, Geometric modelling
Deep Learning (TensorFlow, PyTorch, Keras)
C++ (OpenGL, OpenCV), Python, C#

References

Goodfellow, I., Pouget-Abadie, 2020. Generative adversarial networks. Communications of the ACM, 63(11), pp.139-144.
Narain, R., 2013. Folding and crumpling adaptive sheets. ACM Transactions on Graphics (TOG), 32(4), pp.1-8.
Smith, E.J. ,2017, October. Improved adversarial systems for 3d object generation and reconstruction. In Conference on Robot Learning (pp. 87-96). PMLR.
Xiao, H., 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.
Das, S., 2019. Dewarpnet: Single-image document unwarping with stacked 3d and 2d regression networks.

Apply here.