Thermal Urban Feature Segmentation - Multispectral (RGB + Thermal) UAS-based images from Germany with annotations

The Thermal Urban Feature Segmentation (TUFSeg) dataset consists of annotated and combined RGB and thermal images, acquired by uncrewed aircraft system (UAS). They show various nighttime scenes from two German cities: Munich and Karlsruhe. Owing to the high overlap of 88%, only select images were annotated to prevent duplicate instances.

The raw images were recorded with a 4k normal (RGB) and a FLIR-XT2 (thermal) camera using both DJI M600 Pro and M300 RTK uncrewed aircraft (UA). The RGBs have a resolution of 4000 x 3000 pixels, the thermals 640 x 512. All images were registered to a uniform format (3000 x 3750 pixels) to match thermal image aspect ratio and prevent RGB data loss. The Munich images were recorded during 8 p.m. to 6 a.m. in December 2019 with temperatures ranging from -5 °C to 2 °C, the Karlsruhe ones between 12 a.m. and 3 a.m. in January and March 2022 with temperatures between 0 °C and 3 °C.

The dataset consists of 793 labelled images - 700 from Munich, 93 from Karlsruhe - from 14 UAS flights. A total of 8,010 common urban feature classes are annotated:
buildings - 1,404,
cars (cold) - 2,532,
cars (warm) - 1,036,
... mehr
manholes (cold) - 520,
manholes (warm) - 1,379,
miscellaneous - 81,
people - 275,
street lamps (cold) - 100,
street lamps (warm) - 683.

The data is split into train and test subsets at random with 634 and 159 images, respectively, to match an 80-20 division. All classes and both cities are represented in both sets. All classes and both cities are represented in both splits.

For the annotation of the thermal images, the image processing program VGG Image Annotator from the Visual Geometry Group was used. The urban features were outlined with polygons, placed as closely as possible to but still outside the object in question.