KIT | KIT-Bibliothek | Impressum | Datenschutz

High-Throughput Crowdsourcing Mechanisms for Complex Tasks

Sautter, Guido; Böhm, Klemens

Abstract:

crowdsourcing is popular for large-scale data processing endeav ors that require hu man input. However, working with a large community of users raises new chal lenges. In particular, both possible misjudgment and disho nesty threaten the quality of the results. Common countermeasures are based on redundancy, giving way to a tradeoff between result quality and throughput. Ideally, measures should (1) maintain high throughput and (2) ensure high result quality at the same time. Existing work on crowdsourcing mostly focuses on result quality, paying little attention to throughput or even to that tradeoff. One reason is that the number of tasks (individual atomic units of work) is usually small. A further problem is that the tasks users work on are small as well. In consequence, existing result-improvement mecha nisms do not scale to the number or complexity of tasks that arise, for instance, in proofreading and processing of digitized legacy literature. This paper proposes novel result-improvement mechanisms that (1) are independent of the size and complexity of tasks and (2) allow to trade result quality for throughput to a significant extent. ... mehr


Zugehörige Institution(en) am KIT Institut für Programmstrukturen und Datenorganisation (IPD)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2011
Sprache Englisch
Identifikator ISBN: 978-3-642-24703-3
ISSN: 1611-3349
KITopen-ID: 1000027079
Erschienen in Social Informatics: Proceedings of the Third International Conference (SocInfo 2011), Singapore, October 6-8, 2011. Ed.: A. Datta
Verlag Springer-Verlag
Seiten 240-254
Serie Lecture Notes in Computer Science ; 6984
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page