KIT | KIT-Bibliothek | Impressum | Datenschutz

A comparative Neural Radiance Field (NERF) 3D analsyis of camera poses from HoloLens trajectories and structure from motion

Jäger, Miriam 1; Hübner, Patrick 1; Haitz, Dennis 1; Jutzi, Boris ORCID iD icon 1
1 Institut für Photogrammetrie und Fernerkundung (IPF), Karlsruher Institut für Technologie (KIT)

Abstract:

Neural Radiance Fields (NeRFs) are trained using a set of camera poses and associated images as input to estimate density and color values for each position. The position-dependent density learning is of particular interest for photogrammetry, enabling 3D reconstruction by querying and filtering the NeRF coordinate system based on the object density. While traditional methods like Structure from Motion are commonly used for camera pose calculation in pre-processing for NeRFs, the HoloLens offers an interesting interface for extracting the required input data directly. We present a workflow for high-resolution 3D reconstructions almost directly from HoloLens data using NeRFs. Thereby, different investigations are considered: Internal camera poses from the HoloLens trajectory via a server application, and external camera poses from Structure from Motion, both with an enhanced variant applied through pose refinement. Results show that the internal camera poses lead to NeRF convergence with a PSNR of 25 dB with a simple rotation around the x-axis and enable a 3D reconstruction. Pose refinement enables comparable quality compared to external camera poses, resulting in improved training process with a PSNR of 27 dB and a better 3D reconstruction. ... mehr


Verlagsausgabe §
DOI: 10.5445/IR/1000159527
Veröffentlicht am 16.06.2023
Originalveröffentlichung
DOI: 10.5194/isprs-archives-XLVIII-1-W1-2023-207-2023
Scopus
Zitationen: 4
Dimensions
Zitationen: 9
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Photogrammetrie und Fernerkundung (IPF)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2023
Sprache Englisch
Identifikator ISSN: 2194-9034
KITopen-ID: 1000159527
Erschienen in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Verlag Copernicus Publications
Band XLVIII-1/W1-2023
Seiten 207–213
Vorab online veröffentlicht am 25.05.2023
Schlagwörter Neural Radiance Fields, Microsoft HoloLens, Structure from Motion, Trajectory, 3D Reconstruction, Point Cloud
Nachgewiesen in Scopus
Dimensions
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page