Computer vision-based excavator bucket fill estimation using depth map and faster R-CNN

Excavators are crucial in the construction industry, and developing autonomous excavator systems is vital for enhancing productivity and reducing the reliance on manual labor. Accurate estimation of the volume of the excavator bucket fill is key for monitoring and evaluating system automation performance. This paper presents the use of 2D depth maps as input to a Faster Region Convolutional Neural Network (Faster R-CNN) deep learning model for bucket volume estimation. This structure enables high estimation accuracy while maintaining fast processing speed. An excavator operation monitoring test bench was established, and the datasets used in the study were self-generated for training. A loss function is proposed, combining Cross Entropy with Root Mean Squared Error to improve generalization and precision. Comparative results indicate that the proposed approach achieves 96.91% accuracy in fill factor estimation and predicts in real-time at about 10 fps, highlighting its potential for practical use in automated excavator operations.