Explainable Artificial Intelligence for Biomedical Data: A Systematic Mapping Study

Background: Artificial intelligence (AI) in biomedicine must be explainable to ensure
trust, accountability, and safety. However, most explainable AI (XAI) approaches are
evaluated primarily on technical grounds, with limited systematic assessment and
reliable metrics, hindering translation into practice.
Objective: This study aims to identify and categorize XAI systems for biomedical data
that fulfill criteria for evaluation readiness, defined by output demonstration, input data
availability, and practical applicability.
Methods: A systematic mapping study was conducted using a Scopus search from 2020
to 2025, retrieving 1,178 journal and conference papers. Restricting to 2023 to 2025
yielded 780. Screening identified 27 systems meeting evaluation readiness criteria,
classified by AI evaluation, AI method, agnosticism, data type, input samples, medical
field, medical use case, scope, stage, XAI evaluation, XAI method, and XAI technique.
Results: Imaging-oriented XAI systems dominate, with 63% (n = 17) using radiology or
histopathology pipelines and visual heatmaps such as Grad-CAM and saliency maps.
Tabular (n = 4) and graph-based (n = 3) approaches are less common. ... mehrFeature relevance
methods, especially SHAP (n = 6), prevail. Grad-CAM is frequent in imaging (n = 4), while
graph-based techniques are emerging (n = 4). Evaluation strategies emphasize
functionality-grounded approaches, 52% (n = 14), with fewer application-grounded (n =
11) and rare human-grounded (n = 2) assessments. Most XAI systems provide code and,
in some cases, web applications, enabling reproducibility, e.g., MICA and CIS2MS.
Conclusion: Evaluation-ready XAI systems in biomedicine are scarce, imaging-
dominated, and specialized. Local explanations and architecture modification methods
are underused, and cross-domain benchmarking is essential. The lack of standardized
XAI metrics limits comparability. This mapping offers a starting point for validation. Its
Scopus-only search, single-round coding, and assumed evaluation readiness criteria
motivate broader search, multistage review, and human-grounded research.