Activity analysis of construction equipment using audio signals and support vector machines

In the construction industry, especially for civil infrastructure projects, a large portion of overall project expenses are allocated towards various costs associated with heavy equipment. As a result, continuous tracking and monitoring of tasks performed by construction heavy equipment is vital for project managers and jobsite personnel. The current approaches for automated construction equipment monitoring include both location and action tracking methods. Current construction equipment action recognition and tracking methods can be divided into two major categories: 1) using active sensors such as accelerometers and gyroscopes and 2) implementing computer vision algorithms to extract information by processing images and videos. While both categories have their own advantages, the limitations of each mean that the industry still suffers from the lack of an efficient and automatic solution for the construction equipment activity analysis problem. In this paper we propose an innovative audio-based system for activity analysis (and tracking) of construction heavy equipment. Such equipment usually generates distinct sound patterns while performing certain tasks, and hence audio signal processing could be an alternative solution for solving the activity analysis problem within construction jobsites. The proposed system consists of multiple steps including filtering the audio signals, converting them into time frequency representations, classifying these representations using machine learning techniques (e.g., a support vector machine), and window filtering the output of the classifier to differentiating between different patterns of activities. The proposed audio-based system has been implemented and evaluated using multiple case studies from several construction jobsites and the results demonstrate the potential capabilities of the system in accurately recognizing various actions of construction heavy equipment.