KIT | KIT-Bibliothek | Impressum | Datenschutz

Bring the Environment to Life: A Sonification Module for People with Visual Impairments to Improve Situation Awareness

Constantinescu, Angela; Müller, Karin ORCID iD icon; Haurilet, Monica; Petrausch, Vanessa; Stiefelhagen, Rainer ORCID iD icon


Digital navigation tools for helping people with visual impairments have become increasingly popular in recent years. While conventional navigation solutions give routing instructions to the user, systems such as GoogleMaps, BlindSquare, or Soundscape offer additional information about the surroundings and, thereby, improve the orientation of people with visual impairments. However, these systems only provide information about static environments, while dynamic scenes comprising objects such as bikes, dogs, and persons are not considered. In addition, both the routing and the information about the environment are usually conveyed by speech.
We address this gap and implement a mobile system that combines object identification with a sonification interface. Our system can be used in three different scenarios of macro and micro navigation: orientation, obstacle avoidance, and exploration of known and unknown routes. Our proposed system leverages popular computer vision methods to localize 18 static and dynamic object classes in real-time. At the heart of our system is a mixed reality sonification interface which is adaptable to the user’s needs and is able to transmit the recognized semantic information to the user. ... mehr

Postprint §
DOI: 10.5445/IR/1000125500
Veröffentlicht am 01.11.2021
DOI: 10.1145/3382507.3418874
Zitationen: 12
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Anthropomatik und Robotik (IAR)
Universität Karlsruhe (TH) – Interfakultative Einrichtungen (Interfakultative Einrichtungen)
Karlsruher Institut für Technologie (KIT)
Studienzentrum für Sehgeschädigte (SZS)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2020
Sprache Englisch
Identifikator ISBN: 978-1-4503-7581-8
KITopen-ID: 1000125500
Erschienen in ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction. Ed.: K.Truong
Veranstaltung 22nd International Conference on Multimodal Interaction (ICMI 2020), Online, 25.10.2020 – 29.10.2020
Verlag Association for Computing Machinery (ACM)
Seiten 50–59
Bemerkung zur Veröffentlichung Postprint kann laut Verlag (ACM) ohne Embargo publiziert werden (wurde mit Frau Tobias abgesprochen)
Vorab online veröffentlicht am 31.10.2020
Schlagwörter Assistive technologies; Sonification; Mixed reality; Evaluations of intelligent user interfaces; Object Localization; Computer Vision
Nachgewiesen in Dimensions
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page