Multi-Sensor and Multi-Modal Localization in Indoor Environments on Robotic Platforms

This dissertation presents a detailed exploration of multi-sensor visual odometry systems, with a focus on enhancing robustness in indoor environments. The core contribution lies in the development of an advanced ego-state estimation framework, wherein visual odometry is bolstered by multiple visual sensors to overcome common challenges such as occlusions and textureless surfaces. By addressing frequent loss-of-tracking (LoT) events, the system ensures continuous, reliable localization in complex, cluttered indoor settings, such as households and elderly care facilities.

To further augment situational awareness, sound source localization (SSL) is integrated as a complementary modality. Its fusion with visual data signicantly enhances the robot’s perception of the environment, enabling the detection and identication of objects and events that may be visually occluded or otherwise undetectable. This multi-modal fusion provides a more holistic understanding of the robot’s surroundings, contributing to improved operational reliability in dynamic, human-centered environments.

A key feature of this research is the introduction of IndoorMCD, a novel multisensor benchmark specically designed to evaluate localization performance in indoor environments. ... mehrAdditionally, this work introduces URSim, an online real-time visual simulation framework that enables rigorous testing of multisensor localization systems under various conditions. Extensive experimental
validation, using both real-world scenarios and simulated environments, demonstrates the robustness and fault tolerance of the proposed system.

This research advances the state-of-the-art in robotic perception and indoor localization by providing a multi-modal, fault-tolerant approach to localization, oering valuable contributions to both theoretical understanding and practical application in robotics.