Distributed Feature Selection for Big Data Using Fuzzy Rough Sets

Fuzzy rough-set-based feature selection is an important technique for big data analysis. However, the classic fuzzy rough set algorithm takes all the data correlations into account, which leads to the centralized computing mode, requiring high computing and memory space resources. With the increasing amount of data in the big data era, the centralized server cannot afford the computation of fuzzy rough set. To enable the fuzzy rough set for big data analysis, in this article, we propose the novel distributed fuzzy rough set (DFRS)-based feature selection, which separates and assigns the tasks to multiple nodes for parallel computing. The key challenge is to maintain the global information on each distributed node without conserving the entire fuzzy relation matrix. We tackle this challenge by a dynamic data decomposition algorithm and a data summarization process on each distributed node. Extensive experiments based on multiple real datasets demonstrate that DFRS significantly improves the runtime, and its feature selection accuracy is nearly the same as the traditional centralized computing.