Addressing Low-Quality Domain Knowledge in Knowledge-Guided Machine Learning

Many industries collect vast amounts of data, applying machine learning techniques to derive actionable insights and improve decision-making. However, deploying machine learning models in scientific and engineering contexts presents unique challenges. This is because the available data often may not sufficiently represent the true nature of the underlying phenomena. In consequence, models may not generalize well beyond the training data and may be prone to learning spurious relationships. It is crucial to ensure patterns discovered from data are consistent with relevant domain expertise. Knowledge-guided Machine Learning (KGML, also known as Theory-Guided Data Science, Physics-Guided Machine Learning or Informed Machine Learning) is a subfield of Machine Learning that focuses on systematically integrating domain knowledge into machine learning. Recent studies have shown that even little information integrated in this way can substantially enhance the accuracy, consistency, and generalizability of machine learning models.

However, domain knowledge can sometimes degrade model performance, a phenomenon we refer to as low-quality domain knowledge.
... mehr

Low-quality domain knowledge can arise for several reasons:
(1) Difficult and ineffective process of integrating domain knowledge, due to the interdisciplinary nature of the task and the need for extensive collaboration with domain experts.
(2) Imperfections of domain knowledge, due to difficulties in its collection, definition and representation.
(3) A mismatch between domain knowledge and task-specific requirements, where knowledge originally developed for a different purpose proves to be suboptimal.

This thesis investigates these sources of low-quality domain knowledge and proposes solutions to address them:

(1) To facilitate the process of integrating domain knowledge, we introduce novel evaluation criteria for KGML methods that are crucial for practical collaboration with domain experts but are often overlooked: the amount of domain knowledge required and intellectual effort necessary for implementation. We apply these criteria to compare methods for integrating domain knowledge into machine learning models, using time-series voltage prediction for lithium-ion batteries as a case study. Our findings reveal that while effective KGML methods typically require substantial domain knowledge, they do not always demand significant implementation effort.

(2) To handle imperfect domain knowledge, we demonstrate how to design and evaluate a new KGML approach that addresses two types of imperfections: approximate domain knowledge, when exact measurements are not possible, and incorrect domain knowledge, arising from measurement errors. We formalize a novel form of domain knowledge, commonly encountered in temporal dynamics modeling for engineering systems, and incorporate it through a knowledge-guided loss function. Our results indicate that the proposed method reduces prediction error by up to 40%, while significantly reducing the data requirements for training, and being robust to an incorrect domain knowledge of certain degree and being able to leverage approximate domain knowledge.

(3) To address mismatches between domain knowledge and task-specific knowledge, we systematically study mismatches within Ontology-guided Machine Learning, a subfield of KGML that uses ontological knowledge. We investigate how these mismatches manifest and influence predictions, and propose a framework to quantify them. Applying this framework to image classification and patient health prediction, we find that mismatches can affect substantial portions of the knowledge base, leading to a significant decline in model performance.


These contributions provide a comprehensive framework for addressing common challenges stemming from the low-quality domain knowledge in KGML, enhancing its applicability of KGML in various scientific and engineering domains.