Understanding and Mitigating Corner Cases with Contextual Knowledge in Deep Learning for Automated Driving

Data-driven computer vision has advanced significantly through machine learning approaches that do not require extensive feature engineering. However, deep neural networks can easily produce overconfident predictions when data during inference are not independent and identically distributed as the training set. For safety-critical systems like automated driving, neural networks must accommodate continuously changing input data distributions, such as the shifting weather conditions that can affect the sensors in real-world driving situations. Such properties of data-driven methods inhibit perception models from being deployed in mass-production vehicles, as the driving functions are mandatory to fulfill the quality requirements according to safety regulations and technical standards for automotive software development. In the automotive industry, the aforementioned driving scenarios that lead to inappropriate perception results and, by implication, generate undesirable driving behavior, are often regarded as corner cases. In these cases, vehicles equipped with automated driving functions encounter previously unknown objects or observe abnormal behavior of other traffic participants. ... mehrPrevious work that aimed at providing taxonomies of corner cases did not facilitate classifying such scenarios in this particular field of application and thus solving them. Therefore, this thesis not only strives to provide insights into defining those safety-critical driving situations from the data perspective but also to mitigate them efficiently in advance by systematic processes.

Starting with a preliminary study, it demonstrates that utilizing essential data augmentation methods, which maintain the integrity of the data distribution, can enhance the robustness of data-driven models, while distorted data for the training leads to increasing detection errors and, therefore, more corner cases. With such knowledge, this thesis systematically evaluates the causes roots of corner cases in the context of automated driving functions, unveiling their deep connection to the availability of the data during development.

To mitigate those corner cases without acquiring extensive annotations, approaches are proposed to better utilize readily available prior knowledge, and are evaluated to determine how such knowledge can help deep learning-based image perception systems overcome their limitations dealing with unknown or underrepresented objects.
Current research has not adequately addressed the concept of class hierarchies despite their natural discriminating properties connected with object classes. Starting from the object detection task, leveraging the knowledge learned from some frequent classes also helps to ensure the generalization ability of the neural networks so that a pre-defined hierarchical class taxonomy facilitates detecting previously unknown classes that belong to the same object category. Such a principle also applies to the segmentation tasks, where hierarchical class taxonomies provide valuable insights into semantic segmentation models dealing with previously unknown classes that belong to a known object category. With the help of the more effective feature representations brought by hierarchical taxonomies, learning a novel class with limited training data for corner case elimination demonstrates the potential of leveraging such a priori knowledge. Last but not least, the usage of other publicly available contextual knowledge for efficient domain adaptation is demonstrated with lane detection models. The performance of adapting a lane detection model to a previously unknown domain improves significantly while leveraging only meta information from the target domain.

In summary, this thesis contributes to the systematic understanding of corner cases for automated driving functions based on the availability of data. Leveraging contextual knowledge from various sources improves the models' capability and helps to proactively mitigate possible task-agnostic corner cases due to the knowledge gap without acquiring an extensive amount of data annotation. These observations hold the potential for more robust, reliable, and affordable assisted and automated driving systems.