KIT | KIT-Bibliothek | Impressum | Datenschutz

On the Privacy–Utility Trade-Off in Differentially Private Hierarchical Text Classification

Wunderlich, Dominik; Bernau, Daniel; Aldà, Francesco; Parra-Arnau, Javier 1; Strufe, Thorsten ORCID iD icon 2
1 Institut für Telematik (TM), Karlsruher Institut für Technologie (KIT)
2 Kompetenzzentrum für angewandte Sicherheitstechnologie (KASTEL), Karlsruher Institut für Technologie (KIT)

Abstract:

Hierarchical text classification consists of classifying text documents into a hierarchy of classes and sub-classes. Although Artificial Neural Networks have proved useful to perform this task, unfortunately, they can leak training data information to adversaries due to training data memorization. Using differential privacy during model training can mitigate leakage attacks against trained models, enabling the models to be shared safely at the cost of reduced model accuracy. This work investigates the privacy–utility trade-off in hierarchical text classification with differential privacy guarantees, and it identifies neural network architectures that offer superior trade-offs. To this end, we use a white-box membership inference attack to empirically assess the information leakage of three widely used neural network architectures. We show that large differential privacy parameters already suffice to completely mitigate membership inference attacks, thus resulting only in a moderate decrease in model utility. More specifically, for large datasets with long texts, we observed Transformer-based models to achieve an overall favorable privacy–utility trade-off, while for smaller datasets with shorter texts, convolutional neural networks are preferable.


Verlagsausgabe §
DOI: 10.5445/IR/1000153306
Veröffentlicht am 30.11.2022
Originalveröffentlichung
DOI: 10.3390/app122111177
Scopus
Zitationen: 2
Web of Science
Zitationen: 2
Dimensions
Zitationen: 6
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Telematik (TM)
Kompetenzzentrum für angewandte Sicherheitstechnologie (KASTEL)
Publikationstyp Zeitschriftenaufsatz
Publikationsjahr 2022
Sprache Englisch
Identifikator ISSN: 2076-3417
KITopen-ID: 1000153306
HGF-Programm 46.23 (POF IV, LK 01) Engineering Secure Systems (KASTEL)
Erschienen in Applied Sciences
Verlag MDPI
Band 12
Heft 21
Seiten Art.-Nr.: 11177
Vorab online veröffentlicht am 04.11.2022
Schlagwörter text classification, differential privacy, membership inference
Nachgewiesen in Dimensions
Web of Science
Scopus
Relationen in KITopen
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page