KIT | KIT-Bibliothek | Impressum | Datenschutz

Paintings, not Noise - The Role of Variety and Self-Determination in Labeling Work and its Effects on Psychological and Performance Outcomes

Knaeble, Merlin 1; Nadj, Mario 1; Maedche, Alexander 1
1 Institut für Wirtschaftsinformatik und Marketing (IISM), Karlsruher Institut für Technologie (KIT)


Labeling is critical in creating training datasets for supervised machine learning. It typically requires manual labor, is costly, and not infrequently frustrates the workers involved. Although task variety drives human autonomy and intrinsic motivation, there is little research in this regard in the labeling context. Against this backdrop, we manipulate the presentation sequence of a labeling task in an online experiment using the theoretical lens of self-determination theory to examine workers’ perceptions of task variety, autonomy, and intrinsic motivation, as well as the effects on psychological and performance work outcomes. We rely on 180 crowd workers contributing with group comparisons between three presentation sequences (by image, by label, random) and a mediation path analysis along the phenomena studied. Our results show, how task variety as perceived by workers does not equal mathematical task variety as in randomization. Rather, labeling workers seem to be better accompanied in more structured approaches to the sequence of presentation. We choose a visual metaphor to explain this phenomenon, whereas paintings are but a structured re-arrangement of coloured pixels, as opposed to random noise.

Zugehörige Institution(en) am KIT Institut für Wirtschaftsinformatik und Marketing (IISM)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2022
Sprache Englisch
Identifikator KITopen-ID: 1000146671
Erschienen in CHI 2022 Workshop - Self-Determination Theory in HCI: Shaping a Research Agenda
Veranstaltung Conference on Human Factors in Computing Systems (CHI 2022), New Orleans, LA, USA, 29.04.2022 – 05.05.2022
Externe Relationen Siehe auch
Siehe auch
Schlagwörter interactive labeling, annotation, interactive machine learning, training data, crowd work, crowd-sourcing, task design, self-determination theory
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page