Anomaly Detection in Lidar Data by Combining Supervised and Self-Supervised Methods

To enable safe autonomous driving, a reliable and redundant perception of the environment is
required. In the context of autonomous vehicles, the perception is mainly based on machine learning
models that analyze data from various sensors such as camera, Radio Detection and Ranging
(radar), and Light Detection and Ranging (lidar). Since the performance of the models depends
significantly on the training data used, it is necessary to ensure perception even in situations that
are difficult to analyze and deviate from the training dataset. These situations are called corner
cases or anomalies.
Motivated by the need to detect such situations, this thesis presents a new approach for detecting
anomalies in lidar data by combining Supervised (SV) and Self-Supervised (SSV) models. In particular,
inconsistent point-wise predictions between a SV and a SSV part serve as an indication
of anomalies arising from the models used themselves, e.g., due to lack of knowledge. The SV
part is composed of a SV semantic segmentation model and a SV moving object segmentation
model, which together assign a semantic motion class to each point of the point cloud. ... mehrBased
on the definition of semantic motion classes, a first motion label, denoting whether the point is
static or dynamic, is predicted for each point. The SSV part mainly consists of a SSV scene flow
model and a SSV odometry model and predicts a second motion label for each point. Thereby,
the scene flow model estimates a displacement vector for each point, which, using the odometry
information of the odometry model, represents only a point’s own induced motion. A separate
quantitative analysis of the two parts and a qualitative analysis of the anomaly detection capabilities
by combining the two parts are performed. In the qualitative analysis, the frames are classified
into four main categories, namely correctly consistent, incorrectly consistent, anomalies detected
by the SSV part, and anomalies detected by the SV part. In addition, weaknesses were identified
in both the SV part and the SSV part.