KIT | KIT-Bibliothek | Impressum | Datenschutz

Spectral Reconstruction and Disparity from Spatio-Spectrally Coded Light Fields via Multi-Task Deep Learning

Schambach, Maximilian ORCID iD icon 1; Shi, Jiayang; Heizmann, Michael 1
1 Institut für Industrielle Informationstechnik (IIIT), Karlsruher Institut für Technologie (KIT)


We present a novel method to reconstruct a spectral central view and its aligned disparity map from spatio-spectrally coded light fields. Since we do not reconstruct an intermediate full light field from the coded measurement, we refer to this as principal reconstruction. The coded light fields correspond to those captured by a light field camera in the unfocused design with a spectrally coded microlens array. In this application, the spectrally coded light field camera can be interpreted as a single-shot spectral depth camera.
We investigate several multi-task deep learning methods and propose a new auxiliary loss-based training strategy to enhance the reconstruction performance. The results are evaluated using a synthetic as well as a new real-world spectral light field dataset that we captured using a custom-built camera. The results are compared to state-of-the art compressed sensing reconstruction and disparity estimation.
We achieve a high reconstruction quality for both synthetic and real-world coded light fields. The disparity estimation quality is on par with or even outperforms state-of-the-art disparity estimation from uncoded RGB light fields.

DOI: 10.48550/arXiv.2103.10179
Zugehörige Institution(en) am KIT Institut für Industrielle Informationstechnik (IIIT)
Publikationstyp Forschungsbericht/Preprint
Publikationsdatum 18.03.2021
Sprache Englisch
Identifikator KITopen-ID: 1000143550
Nachgewiesen in arXiv
Relationen in KITopen
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page