Trade-offs Between Privacy-Preserving and Explainable Machine Learning in Healthcare

Background: Machine Learning has enormous potential for applications in various fields. Explainability and privacy are two key questions when training a Machine Learning model especially in critical information infrastructure such as the healthcare sector.
Objective: The goal of this paper is to identify the current state of research and possible trade-offs between explainability and privacy of Machine Learning models. Furthermore, the aim was to identify possible ways of implementing explainability methods in Federated Learning, a privacy-preserving setting.
Methods: First, we have conducted a systematic literature review to identify possible trade-offs. Second, we evaluated and selected methods that one can theoretically implement without risking privacy in a Federated Learning application with a focus on medical image analysis.
Results: Our results show that only a few researchers have so far been discussing possible trade-offs between explainable and privacy-preserving Machine Learning. The three relevant papers show that there is a natural trade-off, and a higher level of explainability can make a model more vulnerable to attacks and therefore have a higher risk of privacy leakage. ... mehrFor our implementation example of explainable Machine Learning methods in Federated Learning we came to the result that it seems to be theoretically possible. Our selected methods are SHapley Additive exPlanations, Gradient-weighted Class Activation Mapping, and Local Interpretable Model-Agnostic Explanations. However, experiments would be necessary to confirm these ideas.
Conclusion: To summarize, we showed that possible trade-offs between explainable and privacy-preserving Machine Learning methods exist, though, is not yet fully discussed in the literature.