Integrating uncertainty quantification into transformer models for large-scale 3D semantic segmentation of urban areas

Large-scale 3D point cloud semantic segmentation is a key capability for remote sensing applications, yet modern deep models often remain overconfident in challenging and ambiguous regions. This work studies uncertainty-aware large-scale 3D semantic segmentation by integrating complementary uncertainty mechanisms into a strong Point Transformer V3 backbone. Specifically, we consider Monte Carlo Dropout (MCD), Deep Sub-Ensembles (DSE) variants, and Test-Time Augmentation (TTA), within a unified training and inference pipeline. We evaluate these methods, together with the baseline model, on two benchmark datasets with distinct characteristics (Hessigheim 3D and DALES) using segmentation metrics, calibration errors, and uncertainty measures, complemented by qualitative visualizations of predicted labels, accuracy maps, uncertainty maps, and accuracy-uncertainty breakdowns. Compared to the baseline, incorporating uncertainty awareness yields gains on the Hessigheim 3D dataset of up to 2.3% in mIoU, 2.3% in mF1, and 2.8% in mAcc. On DALES, the corresponding improvements reach up to 1.9% in mIoU, 1.5% in mF1, and 0.5% in mAcc. Based on the analysis of the final results, the influence of TTA is more evident on segmentation performance and has subtler effects on uncertainty metrics. ... mehrFurthermore, the MCD variants can improve calibration and uncertainty usefulness as the dropout rate increases, at the cost of segmentation accuracy. Finally, DSE variants yield more structured and reliable uncertainty estimates. In particular, we find that the DSE variant with frozen encoder combined with TTA (DSE-FE-TTA) reduces overconfident errors and offers the most robust overall compromise across both datasets, with clearer benefits on the more challenging Hessigheim 3D benchmark in terms of segmentation, calibration, and uncertainty metrics.