KIT | KIT-Bibliothek | Impressum | Datenschutz

Judgment aggregation, discursive dilemma and reflective equilibrium: Neural language models as self-improving doxastic agents

Betz, Gregor 1; Richardson, Kyle
1 Fakultät für Geistes- und Sozialwissenschaften – Institut für Philosophie (PHIL), Karlsruher Institut für Technologie (KIT)

Abstract:

Neural language models (NLMs) are susceptible to producing inconsistent output. This paper proposes a new diagnosis as well as a novel remedy for NLMs' incoherence. We train NLMs on synthetic text corpora that are created by simulating text production in a society. For diagnostic purposes, we explicitly model the individual belief systems of artificial agents (authors) who produce corpus texts. NLMs, trained on those texts, can be shown to aggregate the judgments of individual authors during pre-training according to sentence-wise vote ratios (roughly, reporting frequencies), which inevitably leads to so-called discursive dilemmas: aggregate judgments are inconsistent even though all individual belief states are consistent. As a remedy for such inconsistencies, we develop a self-training procedure—inspired by the concept of reflective equilibrium—that effectively reduces the extent of logical incoherence in a model's belief system, corrects global mis-confidence, and eventually allows the model to settle on a new, epistemically superior belief state. Thus, social choice theory helps to understand why NLMs are prone to produce inconsistencies; epistemology suggests how to get rid of them.


Verlagsausgabe §
DOI: 10.5445/IR/1000153019
Veröffentlicht am 24.11.2022
Originalveröffentlichung
DOI: 10.3389/frai.2022.900943
Scopus
Zitationen: 1
Dimensions
Zitationen: 1
Cover der Publikation
Zugehörige Institution(en) am KIT Fakultät für Geistes- und Sozialwissenschaften – Institut für Philosophie (PHIL)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2022
Sprache Englisch
Identifikator ISSN: 2624-8212
KITopen-ID: 1000153019
Erschienen in Frontiers in Artificial Intelligence
Verlag Frontiers Media SA
Band 5
Seiten Art.-Nr.: 900943
Vorab online veröffentlicht am 18.10.2022
Schlagwörter neural language model (NLM), judgment aggregation, reflective equilibrium, text generation, logical consistency
Nachgewiesen in Dimensions
Scopus
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page