KIT | KIT-Bibliothek | Impressum | Datenschutz
Open Access Logo
Download
Originalveröffentlichung
DOI: 10.3389/fnins.2015.00217
Scopus
Zitationen: 32
Web of Science
Zitationen: 25

Brain-to-text: Decoding spoken phrases from phone representations in the brain

Herff, C.; Heger, D.; de Pesters, A.; Telaar, D.; Brunner, P.; Schalk, G.; Schultz, T.

Abstract:
It has long been speculated whether communication between humans and machines based on natural speech related cortical activity is possible. Over the past decade, studies have suggested that it is feasible to recognize isolated aspects of speech from neural signals, such as auditory features, phones or one of a few isolated words. However, until now it remained an unsolved challenge to decode continuously spoken speech from the neural substrate associated with speech and language processing. Here, we show for the first time that continuously spoken speech can be decoded into the expressed words from intracranial electrocorticographic (ECoG) recordings. Specifically, we implemented a system, which we call Brain-To-Text that models single phones, employs techniques from automatic speech recognition (ASR), and thereby transforms brain activity while speaking into the corresponding textual representation. Our results demonstrate that our system can achieve word error rates as low as 25% and phone error rates below 50%. Additionally, our approach contributes to the current understanding of the neural basis of continuous speech production ... mehr


Zugehörige Institution(en) am KIT Deutsch-Französisches Institut für Automation und Robotik (Dt..-Fr. IAR)
Publikationstyp Zeitschriftenaufsatz
Jahr 2015
Sprache Englisch
Identifikator ISSN: 1662-453X, 1662-4548
URN: urn:nbn:de:swb:90-497506
KITopen ID: 1000049750
Erschienen in Frontiers in neuroscience
Band 9
Heft JUN
Seiten 217
Bemerkung zur Veröffentlichung Gefördert durch den KIT-Publikationsfonds
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft KITopen Landing Page