Dr. Muhammad Zeshan Afzal
Prof. Dr. Didier Stricker
Dr. Alain Pagani
Dr. Gerd Reis
Eric Thil
Keonna Cunningham
Dr. Oliver Wasenmüller
Dr. Gabriele Bleser
Dr. Bruno Mirbach
Dr. Jason Raphael Rambach
Dr. Bertram Taetz
Sk Aziz Ali
Mhd Rashed Al Koutayni
Murad Almadani
Alaa Alshubbak
Yuriy Anisimov
Jilliam Maria Diaz Barros
Ramy Battrawy
Iuliia Brishtel
Hammad Butt
Mahdi Chamseddine
Steve Dias da Cruz
Fangwen Shu
Torben Fetzer
Ahmet Firintepe
Sophie Folawiyo
David Michael Fürst
Kamalveerkaur Garewal
Christiano Couto Gava
Leif Eric Goebel
Tewodros Amberbir Habtegebrial
Simon Häring
Khurram Hashmi
Jigyasa Singh Katrolia
Andreas Kölsch
Onorina Kovalenko
Stephan Krauß
Paul Lesur
Muhammad Jameel Nawaz Malik
Michael Lorenz
Markus Miezal
Mina Ameli
Nareg Minaskan Karabid
Mohammad Minouei
Pramod Murthy
Mathias Musahl
Peter Neigel
Manthan Pancholi
Qinzhuan Qian
Engr. Kumail Raza
Dr. Nadia Robertini
María Alejandra Sánchez Marín
Dr. Kripasindhu Sarkar
Alexander Schäfer
Pascal Schneider
René Schuster
Mohamed Selim
Lukas Stefan Staecker
Dennis Stumpf
Yongzhi Su
Xiaoying Tan
Yaxu Xie
Dr. Vladislav Golyanik
Dr. Aditya Tewari
André Luiz Brandão
Generative View Synthesis: From Single-view Semantics to Novel-view Images
Generative View Synthesis: From Single-view Semantics to Novel-view Images
- Abstract:
-
Generative View Synthesis: From Single-view Semantics to Novel-view Image, Tewodros A. Habtegebrial, Varun Jampani, Orazio Gallo, Didier Stricker . In the Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS-2020) , Virtual.
Abstract:
Content creation, central to applications such as virtual reality, can be a tedious and time-consuming. Recent image synthesis methods simplify this task by offering tools to generate new views from as little as a single input image, or by converting a semantic map into a photorealistic image. We propose to push the envelope further, and introduce Generative View Synthesis (GVS), which can synthesize multiple photorealistic views of a scene given a single semantic map. We show that the sequential application of existing techniques, e.g., semantics-to-image translation followed by monocular view synthesis, fail at capturing the scene's structure. In contrast, we solve the semantics-to-image translation in concert with the estimation of the 3D layout of the scene, thus producing geometrically consistent novel views that preserve semantic structures. We first lift the input 2D semantic map onto a 3D layered representation of the scene in feature space, thereby preserving the semantic labels of 3D geometric structures. We then project the layered features onto the target views to generate the final novel-view images. We verify the strengths of our method and compare it with several advanced baselines on three different datasets. Our approach also allows for style manipulation and image editing operations, such as the addition or removal of objects, with simple manipulations of the input style images and semantic maps respectively.Project page: please more details at the project page gvsnet.github.io.