On-demand monitoring of construction projects through a game-like hybrid application of BIM and machine learning

While unavoidable, inspections, progress monitoring, and comparing as-planned with as-built conditions in construction projects do not readily add tangible intrinsic value to the end-users. In large-scale construction projects, the process of monitoring the implementation of every single part of buildings and reflecting them on the BIM models can become highly labour intensive and error-prone, due to the vast amount of data produced in the form of schedules, reports and photo logs. In order to address the mentioned methodological and technical gap, this paper presents a framework and a proof of concept prototype for on-demand automated simulation of construction projects, integrating some cutting edge IT solutions, namely image processing, machine learning, BIM and Virtual Reality. This study utilised the Unity game engine to integrate data from the original BIM models and the as-built images, which were processed via various computer vision techniques. These methods include object recognition and semantic segmentation for identifying different structural elements through supervised training in order to superimpose the real world images on the as-planned model. The proposed framework leads to an automated update of the 3D virtual environment with states of the construction site. This framework empowers project managers and stockholders with an advanced decision-making tool, highlighting the inconsistencies in an effective manner. This paper contributes to body knowledge by providing a technical exemplar for the integration of ML and image processing approaches with immersive and interactive BIM interfaces, the algorithms and program codes of which can help replicability of these approaches by other scholars.