A Brief Systematization of Explanation-Aware Attacks

Due to the overabundance of trained parameters modern
machine learning models are largely considered black boxes. Explanation
methods aim to shed light on the inner working of such models, and, thus
can serve as debugging tools. However, recent research has demonstrated
that carefully crafted manipulations at the input or the model can suc-
cessfully fool the model and the explanation method. In this work, we
briefly present our systematization of such explanation-aware attacks.
We categorize them according to three distinct attack types, three types
of scopes, and three different capabilities an adversary can have. In our
full paper [12], we further present a hierarchy of robustness notion and
various defensive techniques tailored toward explanation-aware attacks.