AI-based thermal bridge detection of building rooftops on district scale using aerial images

Thermal bridges are weak areas of building envelopes that conduct more heat to the
outside than surrounding envelope areas. They lead to increased energy consumption and the
formation of mold. With a neural network approach, we demonstrate a method of automatically
detecting thermal bridges on building rooftops from panorama drone images of whole city
districts. To train the neural network, we created a dataset including 917 images and 6895
annotations. The images in the dataset contain thermal information for detecting thermal bridges
and a height map for rooftop recognition in addition to regular RGB information. Due to the small
dataset, our approach currently only has an average recall of 9.4% @IoU:0.5-0.95 (14.4% for large
objects). Nevertheless, our approach reliably detects structures only on rooftops and not on other
parts of buildings, without any additional segmentation effort of building parts.