SphereGlue: Learning Keypoint Matching on High Resolution Spherical Images

SphereGlue: Learning Keypoint Matching on High Resolution Spherical Images
Christiano Couto Gava, Vishal Sharbidar Mukunda, Tewodros Amberbir Habtegebrial, Federico Raue, Sebastian Palacio, Andreas Dengel
In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition. International Conference on Computer Vision and Pattern Recognition (CVPR-2023), CVPR 2023 Workshop - Image Matching: Local Features & Beyond, June 18-22, Vancouver, Canada, Pages 6133-6143, IEEE, 6/2023.

Abstract:

Traditionally, spherical keypoint matching has been performed using greedy algorithms, such as Nearest Neighbors (NN) search. NN based algorithms often lead to erroneous or insufficient matches as they fail to leverage global keypoint neighborhood information. Inspired by a recent learned perspective matching approach [53] we introduce SphereGlue: a Graph Neural Network based feature matching for high-resolution spherical images. The proposed model naturally handles the severe distortions resulting from geometric transformations. Rigorous evaluations demonstrate the efficacy of SphereGlue in matching both learned and handcrafted keypoints, on synthetic and real high-resolution spherical images. Moreover, SphereGlue generalizes well to previously unseen real-world and synthetic scenes. Results on camera pose estimation show that SphereGlue can directly replace state-of-the-art matching algorithms, in downstream tasks.