A geometric framework for outlier detection in high-dimensional data

Outlier or anomaly detection is an important task in data analysis. We discuss the problem from a geometrical perspective and provide a framework which exploits the metric structure of a data set. Our approach rests on the manifold assumption, that is, that the observed, nominally high-dimensional data lie on a much lower dimensional manifold and that this intrinsic structure can be inferred with manifold learning methods. We show that exploiting this structure significantly improves the detection of outlying observations in high dimensional data. We also suggest a novel, mathematically precise and widely applicable distinction between distributional and structural outliers based on the geometry and topology of the data manifold that clarifies conceptual ambiguities prevalent throughout the literature. Our experiments focus on functional data as one class of structured high-dimensional data, but the framework we propose is completely general and we include image and graph data applications. Our results show that the outlier structure of highdimensional and non-tabular data can be detected and visualized using manifold learning methods and quantified using standard outlier scoring methods applied to the manifold embedding vectors. This article is categorized under: Technologies > Structure Discovery and Clustering Fundamental Concepts of Data and Knowledge > Data Concepts Technologies > Visualization

Automated summary

Outlier or anomaly detection is a crucial task in data analysis. This paper discusses the problem from a geometrical perspective and proposes a framework that utilizes the metric structure of a dataset. The authors leverage the manifold assumption and show that exploiting this structure significantly enhances the detection of outliers in high-dimensional data. They also introduce a novel and precise distinction between distributional and structural outliers based on the geometry and topology of the data manifold. The experiments demonstrate the effectiveness of manifold learning methods in detecting and visualizing outliers in high-dimensional and non-tabular data.