Towards Robust 6D Pose Tracking for On-Orbit-Servicing with Learned Segmentation and Motion Priors

In this work, we present a robust camera-based approach for tracking the 6D pose of satellites. We build our work on a popular, state-of-the-art visual pose tracker, developed for terrestrial applications. We first analyze its performance and limitations on two novel satellite pose estimation datasets - created with a high-fidelity path-tracing simulator and a real hardware-in-the-loop facility revealing key failures due to inadequate signal from grayscale data, silhouette degradation under orbital lighting, and ambiguities from object symmetries. To overcome these challenges, we introduce two extensions to the core tracking method. First, we use motion priors to constrain the pose optimization, which increases tracking robustness and can mitigate symmetry-induced pose ambiguities. Second, we integrate learning-based image segmentation to handle severe visual variability. This is achieved through Bayesian fusion of classical intensity histograms and learned confidences, extending the formulation to be highly resilient to extreme illumination changes. We evaluate the efficacy of our extensions through comprehensive testing on numerous sequences of varying difficulty, drawn from both rendered and real simulated data. ... mehrThe results demonstrate that our adapted framework provides accurate and reliable 6D pose tracking, showcasing its significant potential to enable safer and autonomous robotic operations for future space missions.