Nonparametric algorithm for identification of outliers in environmental data

Outliers that can significantly affect data analysis are frequently present in environmental data sets. Most methods suggested for the detection of outliers impose restrictions on the distribution of analysed variables. However, in many environmental areas, the observed variable is influenced by a lot of different factors and its distribution is often difficult to find or cannot be estimated. Therefore, an approach for the identification of outliers in environmental time series based on nonparametric statistical techniques is presented. The core principle of the algorithm is to smoothen the data using nonparametric regression with variable bandwidth and subsequently analyse the residuals by nonparametric statistical methods. In the case that the distribution of the analysed variable is normal an efficient statistical method based on normality assumptions is presented as well. The proposed procedure is applied for the identification of outliers in hourly concentrations of particulate matter and verified by simulations. The simulation examples have shown that the presented method is suitable for effective detection of outliers that are deviated at least 7 standard deviations from the mean value of the neighbouring observations. The value of the proposed method is that it reduces the number of observations for manual evaluation and saves the time spent on data control.