KIT | KIT-Bibliothek | Impressum | Datenschutz

Paintings, Not Noise—The Role of Presentation Sequence in Labeling

Knäble, Merlin 1; Nadj, Mario 1; Mädche, Alexander 1
1 Institut für Wirtschaftsinformatik und Marketing (IISM), Karlsruher Institut für Technologie (KIT)

Abstract:

Labeling is critical in creating training datasets for supervised machine learning, and is a common form of crowd work heteromation. It typically requires manual labor, is badly compensated and not infrequently bores the workers involved. Although task variety is known to drive human autonomy and intrinsic motivation, there is little research in this regard in the labeling context. Against this backdrop, we manipulate the presentation sequence of a labeling task in an online experiment and use the theoretical lens of self-determination theory to explain psychological work outcomes and work performance. We rely on 176 crowd workers contributing with group comparisons between three presentation sequences (by label, by image, random) and a mediation path analysis along the phenomena studied. Surprising among our key findings is that the task variety when sorting by label is perceived higher than when sorting by image and the random group. Naturally, one would assume that the random group would be perceived as most varied. We choose a visual metaphor to explain this phenomenon, whereas paintings offer a structured presentation of coloured pixels, as opposed to random noise.


Verlagsausgabe §
DOI: 10.5445/IR/1000169581
Veröffentlicht am 25.03.2024
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Wirtschaftsinformatik und Marketing (IISM)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2024
Sprache Englisch
Identifikator ISSN: 0953-5438, 1873-7951
KITopen-ID: 1000169581
Erschienen in Interacting with Computers
Verlag Oxford University Press (OUP)
Seiten Art.-Nr.: iwae008
Vorab online veröffentlicht am 15.03.2024
Schlagwörter interactive labeling, annotation, interactive machine learning, training data, crowd work, crowdsourcing, task design, variety, self-determination theory
Nachgewiesen in Dimensions
Web of Science
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page