KIT | KIT-Bibliothek | Impressum | Datenschutz

Bring the Environment to Life: A Sonification Module for People with Visual Impairments to Improve Situation Awareness

Constantinescu, Angela; Müller, Karin; Haurilet, Monica; Petrausch, Vanessa; Stiefelhagen, Rainer

Digital navigation tools for helping people with visual impairments have become increasingly popular in recent years. While conventional navigation solutions give routing instructions to the user, systems such as GoogleMaps, BlindSquare, or Soundscape offer additional information about the surroundings and, thereby, improve the orientation of people with visual impairments. However, these systems only provide information about static environments, while dynamic scenes comprising objects such as bikes, dogs, and persons are not considered. In addition, both the routing and the information about the environment are usually conveyed by speech.
We address this gap and implement a mobile system that combines object identification with a sonification interface. Our system can be used in three different scenarios of macro and micro navigation: orientation, obstacle avoidance, and exploration of known and unknown routes. Our proposed system leverages popular computer vision methods to localize 18 static and dynamic object classes in real-time. At the heart of our system is a mixed reality sonification interface which is adaptable to the user’s needs and is able to transmit the recognized semantic information to the user. ... mehr

Open Access Logo

Postprint §
DOI: 10.5445/IR/1000125500
Frei zugänglich ab 01.11.2021
DOI: 10.1145/3382507.3418874
Zugehörige Institution(en) am KIT Studienzentrum für Sehgeschädigte (SZS)
Institut für Anthropomatik und Robotik (IAR)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2020
Sprache Englisch
Identifikator ISBN: 978-1-4503-7581-8
KITopen-ID: 1000125500
Erschienen in ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction. Ed.: K.Truong
Veranstaltung 22nd ACM International Conference on Multimodal Interaction (2020), online, 25.10.2020 – 29.10.2020
Verlag ACM, New York (NY)
Seiten 50–59
Bemerkung zur Veröffentlichung Postprint kann laut Verlag (ACM) ohne Embargo publiziert werden (wurde mit Frau Tobias abgesprochen)
Vorab online veröffentlicht am 31.10.2020
Schlagwörter Assistive technologies; Sonification; Mixed reality; Evaluations of intelligent user interfaces; Object Localization; Computer Vision
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page