KIT | KIT-Bibliothek | Impressum | Datenschutz

Image-to-image translation for enhanced feature matching, image retrieval and visual localization

Mueller, Markus S. 1; Sattler, Thorsten; Pollefeys, Marc; Jutzi, Boris ORCID iD icon 1
1 Institut für Photogrammetrie und Fernerkundung (IPF), Karlsruher Institut für Technologie (KIT)


The performance of machine learning and deep learning algorithms for image analysis depends significantly on the quantity and quality of the training data. The generation of annotated training data is often costly, time-consuming and laborious. Data augmentation is a powerful option to overcome these drawbacks. Therefore, we augment training data by rendering images with arbitrary poses from 3D models to increase the quantity of training images. These training images usually show artifacts and are of limited use for advanced image analysis. Therefore, we propose to use image-to-image translation to transform images from a rendered domain to a captured domain. We show that translated images in the captured domain are of higher quality than the rendered images. Moreover, we demonstrate that image-to-image translation based on rendered 3D models enhances the performance of common computer vision tasks, namely feature matching, image retrieval and visual localization. The experimental results clearly show the enhancement on translated images over rendered images for all investigated tasks. In addition to this, we present the advantages utilizing translated images over exclusively captured images for visual localization.

Verlagsausgabe §
DOI: 10.5445/IR/1000098386
Veröffentlicht am 20.09.2019
DOI: 10.5194/isprs-annals-IV-2-W7-111-2019
Zitationen: 11
Zitationen: 13
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Photogrammetrie und Fernerkundung (IPF)
KIT-Zentrum Klima und Umwelt (ZKU)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2019
Sprache Englisch
Identifikator ISSN: 2194-9050
KITopen-ID: 1000098386
Erschienen in ISPRS annals
Verlag Copernicus Publications
Band IV-2/W7
Seiten 111–119
Vorab online veröffentlicht am 16.09.2019
Schlagwörter Image-to-Image Translation, Convolutional Neural Networks, Generative Adversarial Networks, Data Augmentation, 3D Models, Feature Matching, Image Retrieval, Visual Localization
Nachgewiesen in Dimensions
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page