KIT | KIT-Bibliothek | Impressum | Datenschutz

A Multidimensional Dataset Based on Crowdsourcing for Analyzing and Detecting News Bias

Färber, Michael ORCID iD icon 1; Burkard, Victoria 1; Jatowt, A.; Lim, S. 1
1 Karlsruher Institut für Technologie (KIT)


The automatic detection of bias in news articles can have a high impact on society because undiscovered news bias may influence the political opinions, social views, and emotional feelings of readers. While various analyses and approaches to news bias detection have been proposed, large data sets with rich bias annotations on a fine-grained level are still missing. In this paper, we firstly aggregate the aspects of news bias in related works by proposing a new annotation schema for labeling news bias. This schema covers the overall bias, as well as the bias dimensions (1) hidden assumptions, (2) subjectivity, and (3) representation tendencies. Secondly, we propose a methodology based on crowdsourcing for obtaining a large data set for news bias analysis and identification. We then use our methodology to create a dataset consisting of more than 2,000 sentences annotated with 43,000 bias and bias dimension labels. Thirdly, we perform an in-depth analysis of the collected data. We show that the annotation task is difficult with respect to bias and specific bias dimensions. While crowdworkers' labels of representation tendencies correlate with experts' bias labels for articles, subjectivity and hidden assumptions do not correlate with experts' bias labels and, thus, seem to be less relevant when creating data sets with crowdworkers. ... mehr

Verlagsausgabe §
DOI: 10.5445/IR/1000126464
Veröffentlicht am 20.11.2020
DOI: 10.1145/3340531.3412876
Zitationen: 16
Zitationen: 11
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Angewandte Informatik und Formale Beschreibungsverfahren (AIFB)
Publikationstyp Proceedingsbeitrag
Publikationsjahr 2020
Sprache Englisch
Identifikator ISBN: 978-1-4503-6859-9
KITopen-ID: 1000126464
Erschienen in CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 19-23 October 2020, online
Veranstaltung 29th ACM International Conference on Information and Knowledge Management (CIKM 2020), Online, 19.10.2020 – 23.10.2020
Verlag Association for Computing Machinery (ACM)
Seiten 3007-3014
Bemerkung zur Veröffentlichung Die Veranstaltung fand wegen der Corona-Pandemie als Online-Event statt
Nachgewiesen in Dimensions
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page