KIT | KIT-Bibliothek | Impressum
Open Access Logo
URN: urn:nbn:de:swb:90-440421

Hybrid fNIRS-EEG based classification of auditory and visual perception processes

Putze, F.; Hesslinger, S.; Tse, C.; Huang, Y.; Herff, C.; Guan, C.; Schultz, T.

For multimodal Human-Computer Interaction (HCI), it is very useful to identify the modalities on which the user is currently processing information. This would enable a system to select complementary output modalities to reduce the user's workload. In this paper, we develop a hybrid Brain-Computer Interface (BCI) which uses Electroencephalography (EEG) and functional Near Infrared Spectroscopy (fNIRS) to discriminate and detect visual and auditory stimulus processing. We describe the experimental setup we used for collection of our data corpus with 12 subjects. On this data, we performed cross-validation evaluation, of which we report accuracy for different classification conditions. The results show that the subject-dependent systems achieved a classification accuracy of 97.8% for discriminating visual and auditory perception processes from each other and a classification accuracy of up to 94.8% for detecting modality-specific processes independently of other cognitive activity. The same classification conditions could also be discriminated in a subject-independent fashion with accuracy of up to 94.6 and 86.7%, respectively. We als ... mehr

Zugehörige Institution(en) am KIT Institut für Algorithmen und Kognitive Systeme (IAKS)
Publikationstyp Zeitschriftenaufsatz
Jahr 2014
Sprache Englisch
Identifikator ISSN: 1662-453X
KITopen ID: 1000044042
Erschienen in Frontiers in Neuroscience
Band 8
Seiten 373
Bemerkung zur Veröffentlichung Gefördert durch den KIT-Publikationsfonds
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft KITopen Landing Page