Manifold-based denoising, outlier detection, and dimension reduction algorithm for high-dimensional data

Manifold learning, which has emerged in recent years, plays an increasingly important role in machine learning. However, because inevitable noises and outliers destroy the manifold structure of data, the dimensionality reduction effect of manifold learning will be reduced. Therefore, this paper proposes a denoising algorithm for high-dimensional data based on manifold learning. The algorithm first projects noisy sample vectors onto the local manifold, thereby achieving noise reduction. Then, a statistical analysis of noises is performed to obtain a data boundary. Because all the data come from the same background and obey the same distribution, the sample vectors that are not within the data boundary are marked as outliers, and these outliers are eliminated. Finally, the dimension reduction of the data after noise reduction and outlier detection is performed. Experimental results show that the algorithm can effectively eliminate the interference of noises and outliers in high-dimensional datasets to some extent for manifold learning.

Automated summary

Manifold learning plays an increasingly important role in machine learning, but its dimensionality reduction effect is reduced by inevitable noises and outliers that destroy the manifold structure of data. Therefore, this paper proposes a denoising algorithm based on manifold learning for high-dimensional data. The algorithm first projects noisy sample vectors onto the local manifold to achieve noise reduction. Then, statistical analysis of noises is performed to obtain a data boundary. Outliers, which are sample vectors outside the data boundary, are marked and eliminated. Finally, dimension reduction is performed on the data after noise reduction and outlier detection. Experimental results show that the algorithm can effectively eliminate the interference of noises and outliers in high-dimensional datasets to some extent for manifold learning.