Parallel incremental efficient attribute reduction algorithm based on attribute tree

Attribute reduction is an important application of rough sets. Efficiently reducing massive dynamic data sets quickly has always been a major goal of researchers. Traditional incre-mental methods focus on reduction by updated approximations. However, these methods must evaluate all attributes and repeatedly calculate their importance. When these algo-rithms are applied to large datasets with high time complexity, reducing large decision sys-tems becomes inefficient. We propose an incremental acceleration strategy based on attribute trees to solve this problem. The key step is to cluster all attributes into multiple trees for incremental attribute evaluation. Specifically, we first select the appropriate attri-bute tree for attribute evaluation according to the attribute tree correlation measure to reduce the time complexity. Next, the branch coefficient is added to the stop criterion, increasing with the branch depth and guiding a jump out of the loop after reaching the maximum threshold. This avoids redundant calculation and improves efficiency. Furthermore, we propose an algorithm for incremental attribute reduction based on attri-bute trees using these improvements. Finally, a Spark parallel mechanism is added to par-allelize data processing to implement the parallel incremental efficient attribute reduction based on the attribute tree. Experimental results on the Shuttle dataset show that the time consumption of our algorithm is more than 40% lower than that of the classical IARC algo-rithm while maintaining its good classification performance. In addition, the time is short-ened by more than 87% from the benchmark after adding the Spark parallelizing mechanism.(c) 2022 Elsevier Inc. All rights reserved.

Automated summary

Research on efficient attribute reduction for massive dynamic datasets is important. Traditional incremental methods are inefficient when applied to large datasets. This study proposes an incremental acceleration strategy based on attribute trees, clustering attributes into multiple trees to improve efficiency, and introducing a branch coefficient in the stop criterion to avoid redundant calculations.