Detecting anomalies and de-noising monitoring data from sensors: A smart data approach

When monitoring safety levels in deep pit foundations using sensors, anomalies (e.g., highly correlated variables) and noise (e.g., high dimensionality) exist in the extracted time series data, impacting the ability to assess risks. Our research aims to address the following question: How can we detect anomalies and de-noise monitoring data from sensors in real time to improve its quality and use it to assess geotechnical safety risks? In addressing this research question, we develop a hybrid smart data approach that integrates Extended Isolation Forest and Variational Mode Decomposition models to detect anomalies and de-noise data effectively. We use real-life data obtained from sensors to validate our smart data approach while constructing a deep pit foundation. Our smart data approach can detect anomalies with a root mean square error and signal-to-noise ratio of 0.0389 and 24.09, respectively. To this end, our smart data approach can effectively pre-process data enabling improved decision-making and the management of safety risks.

Automated summary

When monitoring safety levels in deep pit foundations using sensors, the extracted time series data often contain anomalies and noise, which hinders the assessment of risks. In this research, we propose a hybrid smart data approach that combines Extended Isolation Forest and Variational Mode Decomposition models to detect anomalies and de-noise the data effectively. Our approach is validated using real-life sensor data from a deep pit foundation project and achieves a root mean square error of 0.0389 and a signal-to-noise ratio of 24.09 for anomaly detection. Overall, our smart data approach enables improved decision-making and management of safety risks.