Journal
INFORMATION SYSTEMS
Volume 97, Issue -, Pages -Publisher
PERGAMON-ELSEVIER SCIENCE LTD
DOI: 10.1016/j.is.2020.101705
Keywords
Subspace search; Data stream monitoring; Outlier detection
Categories
Funding
- DFG, Germany Research Training Group [2153]
- German Federal Ministry of Education and Research (BMBF) via Software Campus [01IS17042]
Ask authors/readers for more resources
This paper introduces a new method SGMRD that efficiently monitors interesting subspaces in high-dimensional data streams, leading to improved results in downstream data mining tasks.
In the real world, data streams are ubiquitous - think of network traffic or sensor data. Mining patterns, e.g., outliers or clusters, from such data must take place in real time. This is challenging because (1) streams often have high dimensionality, and (2) the data characteristics may change over time. Existing approaches tend to focus on only one aspect, either high dimensionality or the specifics of the streaming setting. For static data, a common approach to deal with high dimensionality - known as subspace search - extracts low-dimensional, Interesting' projections (subspaces), in which patterns are easier to find. In this paper, we address both Challenge (1) and (2) by generalising subspace search to data streams. Our approach, Streaming Greedy Maximum Random Deviation (SGMRD), monitors interesting subspaces in high-dimensional data streams. It leverages novel multivariate dependency estimators and monitoring techniques based on bandit theory. We show that the benefits of SGMRD are twofold: (i) It monitors subspaces efficiently, and (ii) this improves the results of downstream data mining tasks, such as outlier detection. Our experiments, performed against synthetic and real-world data, demonstrate that SGMRD outperforms its competitors by a large margin. (C) 2020 Elsevier Ltd. All rights reserved.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available