Vision Beyond Earth: Synthetic Satellite Data for Neural Perception in Orbit

Satellites have become a critical infrastructure pillar for modern life by supporting key services such as communication, navigation or weather forecasting. Yet, the consistent increase of satellites orbiting Earth brings a number of challenges with it, most importantly the steady rise of the chance of collision between orbiting bodies. At the same time, defunct satellites often remain in orbit as space debris, further adding to the number of items circulating Earth. Therefore, not only active debris removal but also means of extending the life span of a satellite - such as (preventive) maintenance and on-orbit servicing - gain increasing importance. For a system performing these tasks in space, perceptive capabilities are of high importance, such as the initial detection of the target satellite object, its successive tracking to visually follow its trajectory, or even satellite pose estimation. Here, automation is highly beneficial, and computer vision algorithms can greatly contribute - especially neural networks have shown vast advances in recent years for object perception tasks. Yet, the extremely harsh conditions in space create additional challenges, and the performance of detectors or pose estimators in industrial or house-hold applications cannot directly be expected for extra-terrestrial scenarios. ... mehrMost importantly, relevant training data is often not available, and there is a lack of simulators supporting synthetic satellite data generation. In this work, we present Space OAISYS, a major extension of Outdoor Artificial Intelligent SYstems Simulator (OAISYS), supporting vast training data generation for the aforementioned tasks. Space OAISYS simulates satellite missions and creates visual data which can be used in a variety of scenarios. With the simulator one can use an arbitrary satellite object and let it orbit around any kind of celestial body. To use the extension for machine learning task, OAISYS Material Drivers are introduced, which can randomize m...