Journal
AUTOMATION IN CONSTRUCTION
Volume 104, Issue -, Pages 255-264Publisher
ELSEVIER
DOI: 10.1016/j.autcon.2019.03.025
Keywords
Action recognition; Earthmoving excavator; Vision-based; Productivity monitoring; Cycle time; Sequential working pattern; Visual feature; Operation cycle; Convolutional Neural Network (CNN); Long Short Term Memory (LSTM)
Funding
- Seoul National University
Ask authors/readers for more resources
This paper proposes a vision-based action recognition framework that considers the sequential working patterns of earthmoving excavators for automated cycle time and productivity analysis. The sequential patterns of visual features and operation cycles are incorporated into the action recognition framework, which includes three main processes: excavator detection, excavator tracking, and excavator action recognition. The first process uses a pre-trained detector to localize earthmoving excavators on sites from all of captured images. Next, the detection results are associated and the excavators are tracked by Tracking-Learning-Detection algorithm. Lastly, to recognize the operation types of the excavators, their sequential patterns of visual features and operation cycles are modeled and trained with a hybrid deep learning algorithm, i.e., Convolutional Neural Networks and Double-layer Long Short Term Memory. Three experiments were performed to validate the proposed framework and confirm the positive effects of sequential modeling: (1) an experiment without sequential pattern analysis, (2) an experiment with the sequential pattern analysis of visual features, and (3) an experiment with the sequential pattern analysis of visual features and operation cycles. In the experiments, the research team used a total of 72,365 images collected from actual earthmoving sites. The average accuracies of the excavator action recognition in the three cases were 79.8%, 90.9%, and 93.8% respectively. The results demonstrated the applicability of the proposed framework and the significant positive impacts of sequential pattern modeling on the recognition performance. This research contributed to the identification of critical elements that explain sequential working patterns and to the development of a novel vision-based action recognition framework. In addition, the findings of this study can help to automate cycle time analysis and productivity monitoring of earthmoving excavators.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available