The Network of Excellence dAIEDGE organized two successful workshops at the High Performance Embedded Architectures and Compilers Conference HiPEAC in Barcelona, Spain on January 20-22, 2024.

The workshop “Excellence in AI and Edge Computing” held on Monday, 20 January, was  dedicated to exploring the frontiers of the AI and edge computing synergy, focusing on the development of scalable and robust AI systems. It opened with a keynote by Dr. Alireza Dehghani from the Irish National Centre for Applied AI CeADAR, followed by the presentation of 15 scientific papers and a demo session. The workshop closed with a matchmaking session about “Industrial investment priorities for successful AI at the Edge in Europe”.

The workshop “Charting the Future of Edge AI: Functional and Non-Functional Requirements in the Age of Generative AI” held on Tuesday, 21 January, was organized by Ovidiu Vermesan (SINTEF) and Alain Pagani (DFKI). It provided a comprehensive platform for stakeholders to exchange ideas, share experiences, and collaborate on advancing edge AI technologies, with a special focus on addressing functional and non-functional requirements defined based on system engineering principles. The three sessions covered “Requirements in the Generative AI Era”, “Technology developments” and “Application developments” with presentations, followed by a panel discussion on Edge AI Future Trends.

Coordinator: Dr. Alain Pagani

Project Manager : Dr. Mohamed Selim

Our paper “G3FA: Geometry-guided GAN for Face Animation” was presented at the British Machine Vision Conference (BMVC) 2024 by Alireza Javanmardi from the Augmented Vision Department DFKI!

The British Machine Vision Conference (BMVC) is one of the major international conferences on computer vision and related areas and is organised by the British Machine Vision Association (BMVA). This year it was held in Glasgow from November 25^th to 28^th.

Our work presents a method for real-time face re-enactment from one single source image and addresses a key limitation of GAN-based face animation: geometric consistency.

G3FA enhances realism, especially in challenging head poses, by:

• Utilizing implicit 3D supervision through inverse rendering to extract depth and normal maps from 2D images, guiding the generator towards greater accuracy.
• Employing an ensemble of discriminators to incorporate the 3D properties, improving the generator’s understanding of human head structure.
• Leveraging face volume rendering with orthogonal ray sampling and volume rendering for high-quality image synthesis.

Check out the paper:

G3FA: Geometry-guided GAN for Face Animation, Alireza Javanmardi, Alain Pagani, Didier Stricker, British Machine Vision Conference (BMVC), 2024

Link to the paper

Code is available at github.com/dfki-av/G3FA

Contact: Alireza Javanmardi, Alain Pagani

The Network of Excellence dAIEDGE co-organized the  European Conference on EDGE AI Technologies and Applications (EEAI 2024) in Cagliary, Italy on October 21-23, 2024.

Over the three days, more than 50 presenters took part in 12 keynotes and parallel sessions, addressing more than 20 topics, where participants discussed how Edge AI is redefining the future of technological applications and systems.

dAIEDGE Partners contributed to the EEAI programme with presentations that offered a practical and innovative view of Edge AI in action, highlighting its transformative potential in sectors such as sustainability, energy and security:

FPGA Acceleration for SAR Imagery-Based Sea Ice Mapping: This presentation, by Rashed Al Koutayni (DFKI), addressed the use of FPGA accelerators to improve the speed and accuracy of sea ice mapping using synthetic aperture radar (SAR) imagery.

Functional and Non-Functional Requirements in Edge AI Systems: Ovidiu Vermesan (SINTEF, Norway) and Alain Pagani (DFKI, Germany) discussed the challenges associated with functional and non-functional requirements in Edge AI systems, addressing both operational needs and ethical and legal constraints.

Legal Requirements for Trustworthy Edge AI: Lydia Belkadi from KU Leuven addressed the regulations needed to ensure trustworthy edge AI solutions

The event allowed participants not only to learn from leading experts, but also to network and connect with the best in the industry, strengthening a community of experts and practitioners who are driving innovation in Edge AI.

Coordinator: Dr. Alain Pagani

Project Manager: Dr. Mohamed Selim

The researchers of the Augmented Vision department presented their latest research activities in the field of eXtended Reality at the EuroXR conference in Athens, from November 27^th to 29^th, 2024.

The European Association for eXtended Reality (EuroXR) Conference is the ultimate gathering for Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). This year, the conference was co-organized by the Institute of Communication and Computer System (ICCS) of the National Technical University of Athens (NTUA).

This year, a joint event on Next Generation XR was co-organized by six European Projects, among them CORTEX2, LUMINOUS and SHARESPACE, all coordinated by the Augmented Vision department at DFKI.

• Alain Pagani presented the recent work on cooperative telepresence in the project CORTEX2
• Michael Lorenz talked about social XR in SHARESPACE
• Zeshan Afzal featured the use of language models for XR in the project LUMINOUS

The coordinators took part in a panel discussion about the “Future of XR”, in which they could share their view about the next steps in XR research and development.

In addition, the AV group presented two application papers and one demonstrator, which attracted the interest of the conference participants

• “Augmented telecommunication in factory setting”, Narek Minaskan, Bastian Krayer, Alain Pagani, Didier Stricker, EuroXR 2024
• “AR Assistant for Pruning of Grapevines and Fruit Trees”, Mariia Podguzova, Simon Haering, Jamiu Ojeleye, Stephan Krauss, Didier Stricker, EuroXR 2024
• “Industrial remote cooperation (Demo)”, Bastian Krayer, Narek Minaskan, Alain Pagani, Didier Stricker, EuroXR

The paper “AR Assistant for Pruning of Grapevines and Fruit Trees” won the EuroXR 2024 Best Application Paper award – Congratulations to the authors!

 
Contact: Dr. Alain Pagani

Dr. Jason Rambach, gave a talk on “Building Virtual Worlds with 3D Sensing and AI” at the NEM Summit 2024 in Brussels on the 23.10.2024. The presentation was part of the AI for Virtual Worlds session organized by Future Artificial Intelligence Research (FAIR). The presentation with results from the Project HumanTech and various XR projects based on object pose estimation technology from DFKI, was followed by an exciting panel discussion.                                                

Contact: Dr. Jason Rambach

The HumanTech project, under the coordination of DFKI Augmented Vision – Dr. Jason Rambach, has reached another key milestone with a highly productive General Assembly Meeting held from 5 to 6 September 2024 at the Technological University of the Shannon (TUS), Athlone Campus, Ireland. As the project is on its final path toward completion next year, this meeting marked an important step in aligning strategies and objectives for the remaining phases.

The overall status of the project shows that everything is progressing as planned, with the pilots well on track and providing a solid foundation for both achieving the current project goals and supporting future research and development initiatives. Additionally, participants enjoyed a guided tour and live demonstration at TUS’s state-of-the-art robotics department, showcasing cutting-edge technology that will play a crucial role in advancing AI applications in the construction industry.

For more on HumanTech: https://humantech-horizon.eu/news/

Contact: Dr. Jason Rambach

We are glad to announce that our paper, Text2CAD: Generating Sequential CAD Models from Beginner-to-Expert Level Text Prompts has been accepted as a spotlight presentation at the Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS) 2024. The project page is available here. Our research introduces the first AI framework to generate sequential CAD designs from text prompts that include numerical parameters. We look forward to presenting our work in Vancouver, Canada.

Contact: muhammad_zeshan.afzal@dfki.de , mohammad.khan@dfki.de

Dr. Jason Rambach, coordinator of the EU Horizon Project HumanTech , participated in the “Digital Twins for Sustainable Construction” session at the Sustainable Places conference in Luxembourg (23.9-27.9) organized by projects of the Tech4Construction cluster.

Dr. Rambach presented results from the Project HumanTech, focusing on the automated Scan-to-BIM problem. In addition, he was a panel member for the ensuing discussion together with coordinator and members of other ongoing projects in construction and building management such as RobetArme, Beeyonders, InCube and Reincarnate.

Workshop schedule: https://www.sustainableplaces.eu/sp2024/exploring-digital-twins-machine-human-collaboration-and-sustainable-construction/

HumanTech project:     https://humantech-horizon.eu/                                                           

Contact: Dr. Jason Rambach

The Network of Excellence dAIEDGE held its General Assembly meeting on September 5^th and 6th, looking back at the achievements of the last 12 months, and preparing the next steps of the network! It’s been two days full of inspiration and breakthroughs in Heraklion, Crete!

The 36 dAIEDGE partners gathered to share achievements and define the roadmap towards our big goal: To make Europe a world leader in AI and data economy!

During this event, we reinforced our strategic partnerships and strengthened our joint commitment to take technological innovation to another level.

A big thank to our partner ICS-FORTH for being exceptional hosts and making this assembly an unforgettable experience.

We continue to move forward together towards a bright future of innovation and growth!

Coordinator: Dr. Alain Pagani

Project Manager : Dr. Mohamed Selim

We are very happy that our paper “CLEO: Continual Learning of Evolving Ontologies” has been accepted to the European Conference on Computer Vision (ECCV). A preprint of our work can be found here. A short video is available on YouTube. The paper is a joint work with ZF Friedrichshafen AG and describes some results of our collaboration.

We are proud to announce that our paper “RMS-FlowNet++: Efficient and Robust Multi-scale Scene Flow Estimation for Large-Scale Point Clouds” by Ramy Battrawy, René Schuster, and Didier Stricker has been published in the International Journal of Computer Vision (IJCV). The online version of the paper can be found here a preprint is available here. The paper extends our previous work on efficient scene flow estimation in dense point clouds that has been published at ICRA 2022.

The team of the EU Horizon Project HumanTech , consisting of Mahdi Chamseddine and Dr. Jason Rambach from DFKI Augmented Vision as well as Fabian Kaufmann from RPTU Kaiserslautern – department of Civil Engineering, received the 3^rd place prize in the Scan-to-BIM challenge of the (Computer Vision in the Built Environment) CV4_AEC Workshop of the CVPR 2024 conference.

On the 18.6, the team presented their solution and results as part of the workshop program. Scan-to-BIM solutions are of great importance for the construction community as they automate the generation of as-built models of buildings from 3D scans, and can be used for quality monitoring, robotic task planning and XR visualization, among other applications.

HumanTech project:     https://humantech-horizon.eu/

CV4AEC Workshop page: https://cv4aec.github.io/                                                     

Contact: Dr. Jason Rambach  , Mahdi Chamseddine

The 6th workshop on Safe Artificial Intelligence for All Domains was held this year in conjunction with CVPR in Seattle. René Schuster is a permanent member of the program committee since 2020.

We are excited to share that the Augmented Vision group got two papers accepted at the IEEE conference on Automatic Face and Gesture Recognition (FG 2024), the premier international forum for research in image and video-based face, gesture, and body movement recognition. FG 2024 took place in Istanbul, Turkey, from the 27^th to 31^st of May. 

Homepage: https://fg2024.ieee-biometrics.org/ 

The accepted papers are: 

1. SynthSL: Expressive Humans for Sign Language Image Synthesis 

Jilliam Maria Diaz Barros, Chen-Yu Wang, Jameel Malik, Abdalla Elsayed Abdou Elsayed Mohamed Arafa, Didier Stricker 

2. In-Domain Inversion for Improved 3D Face Alignment on Asymmetrical Expressions 

Jilliam Maria Diaz Barros, Jason Raphael Rambach, Pramod Murthy, Didier Stricker 

Contact: Jilliam Diaz Barros, Dr. Jason Rambach 

The 4^th IEEE Workshop On AI Hardware: Test, Reliability And Security (AI-TREATS) held on May 23^th and 24^th 2024 was a great success!

The event, co-located with the 29th IEEE European Test Symposium, explored the recent advances in edge AI under the aspects of security, trust and testing. The workshop gathered many researchers with 15 presented papers, and a keynote by Prof. Yanjing Li (University of Chicago, USA).

On the opening day, Alain Pagani participated to a Panel Discussion about Networks of Excellence on Edge AI in Europe. Together with Mateo Sonza Reorda (Politecnico di Torino Italy, FAIR project) and Ovidiu Vermesan (SINTEF, Norway, EdgeAI project), he could present dAIEDGE  and the main aspects of the Network of Excellence on Edge AI.

The workshop chaired by Annachiara Ruospo (Politecnico di Torino, Italy) and Haralampos Stratigopoulos (Sorbonne Universtié, France) was co-sponsored and co-organised by FAIR, dAIEDGE and EdgeAI

Many thanks to the all participants!

Dr. Jason Rambach, coordinator of the EU Horizon Project HumanTech , organized the 2nd workshop on “AI and Robotics in Construction” at the European Robotics Forum 2024 in Rimini, Italy (13.3-15.3) in cooperation with the construction Robotics projects Beeyonders and RobetArme and the Tech4Construction cluster.

The workshop included presentations on the current state of the 3 organizing projects (HumanTech, Beeyonders and RobetArme) from their coordinators, followed by technical presentations from project partners (KU Leuven, SINTEF, ITAINNOVA) on topics such as robotic vision and navigation and human-robot interaction for construction, and closing with a user evaluation insights presentation by the German Federal Institute for Occupational Safety and Health (BAUA). The presentations were followed by a very interesting round table discussion on the challenges of robotic projects in construction.

Workshop schedule: https://humantech-horizon.eu/wp-content/uploads/2024/02/AI-and-Robotics-in-Construction_ERF2024_Brochure.pdf

HumanTech project: https://humantech-horizon.eu/                                                           

Contact: Dr. Jason Rambach

We are proud to announce that the researchers of the department Augmented Vision will present 6 papers at the upcoming CVPR conference taking place Mon Jun 17th through Fri Jun 21st, 2024 at the Seattle Convention Center, Seattle, USA.

The CVPR conference is the premier international conference in computer vision and pattern recognition.

The 6 papers are:

MiKASA: Multi-Key-Anchor Scene-Aware Transformer for 3D Visual Grounding
Chun-Peng Chang, Shaoxiang Wang, Alain Pagani, Didier Stricker

HiPose: Hierarchical Binary Surface Encoding and Correspondence Pruning for RGB-D 6DoF Object Pose Estimation
Yongliang Lin, Yongzhi Su, Praveen Nathan, Sandeep Inuganti, Yan Di, Martin Sundermeyer, Fabian Manhardt, Didier Stricker, Jason Rambach, Yu Zhang

SG-PGM: Partial Graph Matching Network with Semantic Geometric Fusion for 3D Scene Graph Alignment and Its Downstream Tasks
Yaxu Xie, Alain Pagani, Didier Stricker

Sparse Semi-Detr: Sparse Learnable Queries for Semi-Supervised Object Detection
Tahira Shehzadi, Khurram Azeem Hashmi, Didier Stricker, Muhammad Zeshan Afzal

CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided Attention
Mohammad Sadil Khan, Elona Dupont, Sk Aziz Ali, Kseniya Cherenkova, Anis Kacem, Djamila Aouada

EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams
Christen Millerdurai, Hiroyasu Akada, Jian Wang, Diogo Luvizon, Christian Theobalt, Vladislav Golyanik

Congratulations to the authors for this great achievement!

DFKI Augmented Vision researcher Mahdi Chamseddine received the Best Industrial Paper award at International Conference on Pattern Recognition Applications and Methods (ICPRAM) 2024 for the paper:

CaRaCTO: Robust Camera-Radar Extrinsic Calibration with Triple Constraint Optimization. Mahdi Chamseddine, Jason Rambach, Didier Stricker, ICPRAM 2024

The paper introduces a simplified and improved extrinisic calibration approach for camera-radar systems without the need for external sensing and with additional optimization constraints for added robustness.

DFKI Augmented Vision presented 3 other papers at ICPRAM 2024.

Contact: Mahdi Chamseddine, Dr. Jason Rambach

The HumanTech project, coordinated by DFKI Augmented Vision – Dr. Jason Rambach, has reached an important milestone: A highly successfully Mid-Term Review Meeting!

From 22 to 24 January 2024, representatives from the 21 partner organisations that comprise the consortium gathered in Zurich, Switzerland (hosted by the partner Implenia) to comprehensively review its midterm progress since starting in June 2022. The process has helped the consortium to align priorities to further its mission — to achieve breakthroughs in cutting-edge technologies, contributing to a safer, more efficient and digitized European construction industry. The review meeting consisted of a construction site visit, presentations of the project progress for all work packages and an exciting demo event with live HumanTech technologies.

More on HumanTech: https://humantech-horizon.eu/news/

Contact: Dr. Jason Rambach

We are happy to announce that the Augmented Vision group presented 2 papers in the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) that took place from the 4^th -8^th January 2024 in Waikoloa, Hawaii.

Homepage: https://wacv2024.thecvf.com/   

The 2 accepted papers are:

1. Single Frame Semantic Segmentation Using Multi-Modal Spherical Images
   • Suresh Guttikonda, Jason Raphael Rambach
2. SphereCraft: A Dataset for Spherical Keypoint Detection, Matching and Camera Pose Estimation
   • Christiano Couto Gava, Yunmin Cho, Federico Raue, Sebastian Palacio, Alain Pagani, Andreas Dengel

Contact: Dr. Jason Rambach, Dr. Alain Pagani