Quantifying Domain-Application Knowledge Mismatch in Ontology-Guided Machine Learning

In this work, we study the critical issue of knowledge mismatch in ontology-guided machine learning (OGML), specifically between domain ontologies and application ontologies. Such mismatches may arise when OGML uses ontological knowledge that was originally created for different purposes. Even if ontological knowledge improves the overall OGML performance, mismatches can lead to reduced performance on specific data subsets compared to machine-learning models without ontological knowledge. We propose a framework to quantify this mismatch and identify the specific parts of the ontology that contribute to it. To demonstrate the framework’s effectiveness, we apply it to two common OGML application areas: image classification and patient health prediction. Our findings reveal that domain-application mismatches are widespread across various OGML approaches, machine-learning model architectures, datasets, and prediction tasks, and can impact up to 40% of unique domain concepts in the datase ts. We also explore the potential root causes of these mismatches and discuss strategies to address them.