Automated identification of quoted actors in journalistic articles

Automated Identification of Quoted Actors in Journalistic Articles
"Who gets to speak in the news is crucial for the shaping of the news."(Beckers & Van Aelst, 2019) Identifying actors who are actively contributing to mediated public discourses on societally relevant issues is crucial for examining the degree of deliberation in democratic decision-making processes (Habermas, 1992). Hence, this study evaluates a fully automated procedure for identifying such active actors that are directly or indirectly quoted in journalistic articles.
The technical developed pipeline comprises two components. First, all entities in the text are identified using a Named Entity Recognition (NER) procedure, which was validated in a prior study (Buz et al., 2022). Second, through a combination of rule-based filters and targeted prompts sent to ChatGPT-4o with a temperature of 0.2 for each extracted entity, it is automatically checked whether they are 1) the author of the article, 2) a real person, and 3) directly or indirectly quoted. Finally, 4) duplicates are excluded from the analysis.
The ChatGPT classification of actors was compared against manual coding of a diverse sample of n = 524 news articles covering science and technology issues. ... mehrThe articles were drawn from six German newspapers and included 5,042 named entities, of which 1,130 distinct actors were directly or indirectly quoted. The adjusted confusion matrix (TP = 887, FP = 198, FN = 103, TN = 3854) yields a precision metric of .818, a recall of .896 and a F1 score of .855.
The automatic identification of active actors constitutes a foundational step for many downstream analytical tasks—such as automated statement analysis, examinations of actor constellations, or topic-specific discourse modeling in large-scale text corpora. With our automated approach we contribute to raising efficiency in preliminary classification procedures enabling either an upscaling of basic actor analyses or the release of time and personal resources that can be used for more in-depth consecutive examinations.
The data and script underlying this study are publicly available on GitLab and can be accessed via the following link: https://gitlab.com/wisskomm-in-digitalen-medien/ automated-identification-of-quoted-actors-in-journalistic-articles.