KIT | KIT-Bibliothek | Impressum | Datenschutz

Toward User-adaptive Interactive Labeling on Crowdsourcing Platforms

Knaeble, Merlin 1; Nadj, Mario 1; Maedche, Alexander 1; Loewe, Nico 1
1 Institut für Wirtschaftsinformatik und Marketing (IISM), Karlsruher Institut für Technologie (KIT)


As machine learning continues to grow in popularity, so does the need for labeled training data. Crowd workers often have to tag, label, or annotate these datasets in the course of a labour-intensive, monotonous, and error-prone process that can even be frustrating. However, current task and system designs typically disregard worker-centric issues. In this vision statement, we argue that given the rising human-AI interaction in crowd work, further attention needs to be paid to the design of labeling systems in this regard. Specifically, we see the need for platforms to adapt dynamically to affective-cognitive states of crowd workers based on different types of data (i.e., physiological, behavioral or self-reported). A platform that is considerate to its crowd workers should be able to adapt to such states on an individual level, for instance, by suggesting currently fitting tasks. As a conclusion, we call for interdisciplinary research to make this vision a reality.

Zugehörige Institution(en) am KIT Institut für Wirtschaftsinformatik und Marketing (IISM)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2022
Sprache Englisch
Identifikator KITopen-ID: 1000146670
Erschienen in CHI 2022 Workshop - REGROW: Reimagining Global Crowdsourcing for Better Human-AI Collaboration
Veranstaltung Conference on Human Factors in Computing Systems (CHI 2022), New Orleans, LA, USA, 29.04.2022 – 05.05.2022
Externe Relationen Siehe auch
Schlagwörter crowdsourcing, crowd work, interactive labeling, user-adaption
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page