KIT | KIT-Bibliothek | Impressum | Datenschutz

Evaluating Explanation Methods for Deep Learning in Computer Security

Warnecke, Alexander; Arp, Daniel; Wressnegger, Christian; Rieck, Konrad


Deep learning is increasingly used as a building block of security systems. Unfortunately, neural networks are hard to interpret and typically opaque to the practitioner. The machine learning community has started to address this problem by developing methods for explaining the predictions of neural networks. While several of these approaches have been successfully applied in the area of computer vision, their application in security has received little attention so far. It is an open question which explanation methods are appropriate for computer security and what requirements they need to satisfy.
In this paper, we introduce criteria for comparing and evaluating explanation methods in the context of computer security. These cover general properties, such as the accuracy of explanations, as well as security-focused aspects, such as the completeness, efficiency, and robustness. Based on our criteria, we investigate six popular explanation methods and assess their utility in security systems for malware detection and vulnerability discovery. We observe significant differences between the methods and build on these to derive general recommendations for selecting and applying explanation methods in computer security.

Zugehörige Institution(en) am KIT Institut für Theoretische Informatik (ITI)
Publikationstyp Proceedingsbeitrag
Publikationsdatum 09.09.2020
Sprache Englisch
Identifikator KITopen-ID: 1000121181
Erschienen in Proceedings of the 5th IEEE European Symposium on Security and Privacy (EuroS&P)
Veranstaltung 5th IEEE European Symposium on Security and Privacy (EuroSP 2020), Online, 07.09.2020 – 11.09.2020
Bemerkung zur Veröffentlichung Conference is being rescheduled to an all-digital conference on September 7-11, 2020
Vorab online veröffentlicht am 07.09.2020
Schlagwörter explainability, machine learning
Nachgewiesen in arXiv
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page