Computational, Label, and Data Efficiency in Deep Learning for Sparse 3D Data

The deep learning technology has made fast progress in recent years. It is widely applied to sparse 3D data to perform challenging tasks, \eg, 3D object detection and semantic segmentation.
However, the high performance of deep learning comes with high costs, including computational costs and the effort to capture and label data.
This thesis investigates and improves the efficiency of deep learning for sparse 3D data to overcome the obstacles to the further development of this technology.

For better computational efficiency, a depth map-based 3D object detector is introduced. Also, transformer-based models are explored to process point clouds that cannot be represented as depth maps. The proposed novel architectures achieve competitive performance with lower computational costs than existing methods.

Also, to reduce the dependence on labeled data and improve label efficiency, this thesis researches self-supervised pre-training, which only requires unlabeled data. Specifically, it provides a closer look at invariance-based contrastive learning using 3D data and the masked auto-encoder for point clouds.
Compared to directly training neural networks on target datasets, self-supervised pre-training brings a significant performance boost without additional labels. ... mehr

Furthermore, synthetic data generation for pre-training is investigated to reduce the effort of capturing real-world 3D data and improve data efficiency. Instead of applying sophisticated simulation, this thesis generates data using a fully randomized approach. The generated synthetic data perform well with different neural networks and pre-training methods. Also, the performance is competitive compared to real-world data.