KIT | KIT-Bibliothek | Impressum | Datenschutz

Accelerating Deductive Coding of Qualitative Data: An Experimental Study on the Applicability of Crowdsourcing

Haug, Saskia ORCID iD icon; Rietz, Tim; Maedche, Alexander

Abstract (englisch):

While qualitative research can produce a rich understanding of peoples’ mind, it requires an essential and strenuous data annotation process known as coding. Coding can be repetitive and time-consuming, particularly for large datasets. Crowdsourcing provides flexible access to workers all around the world, however, researchers remain doubtful about its applicability for coding. In this study, we present an interactive coding system to support crowdsourced deductive coding of semi-structured qualitative data. Through an empirical evaluation on Amazon Mechanical Turk, we assess both the quality and the reliability of crowd-support for coding. Our results show that non-expert coders provide reliable results using our system. The crowd reached a substantial agreement of up to 91% with the coding provided by experts. Our results indicate that crowdsourced coding is an applicable strategy for accelerating a strenuous task. Additionally, we present implications of crowdsourcing to reduce biases in the interpretation of qualitative data.

Postprint §
DOI: 10.5445/IR/1000136239
Veröffentlicht am 01.10.2022
DOI: 10.1145/3473856.3473873
Zitationen: 1
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Wirtschaftsinformatik und Marketing (IISM)
Publikationstyp Proceedingsbeitrag
Publikationsmonat/-jahr 09.2021
Sprache Englisch
Identifikator ISBN: 978-1-4503-8645-6
KITopen-ID: 1000136239
Erschienen in Mensch und Computer 2021, MuC '21, 5.-8.9.2021 Ingolstadt
Veranstaltung Mensch und Computer (MuC 2021), Online, 05.09.2021 – 08.09.2021
Verlag Association for Computing Machinery (ACM)
Seiten 432–443
Schlagwörter Crowdsourcing, Coding, Qualitative Data, Empirical Evaluation
Nachgewiesen in Scopus
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page