Snap, Segment, Deploy: A Visual Data and Detection Pipeline for Wearable Industrial Assistants

Industrial assembly requires rapid adaptation to complex procedures under constrained computing, connectivity, and privacy conditions, rendering cloud-based solutions impractical. We present an on-device assistant for real-time, semi-hands-free guidance, integrating lightweight detection, speech recognition, and retrieval-augmented response generation. To enable scalable training without manual labeling, we construct the Gear8 dataset via an automated pipeline and introduce a two-stage refinement strategy to enhance robustness against domain shift. Experiments show improved generalization under diverse corruptions. User studies confirm notable gains in efficiency and error reduction, underscoring the system’s suitability for real-world industrial deployment. The Gear8 dataset, models, and code are publicly available at: https://github.com/Kratos-Wen/Gear8.