4.7 Article

ReliefE: feature ranking in high-dimensional spaces via manifold embeddings

Journal

MACHINE LEARNING
Volume 111, Issue 1, Pages 273-317

Publisher

SPRINGER
DOI: 10.1007/s10994-021-05998-5

Keywords

Feature ranking; Representation learning; Relief

Funding

  1. Slovenian Research Agency (ARRS) [P2-0103]
  2. ERC Complementary Scheme [N2-0078]
  3. TAILOR - EU [952215]
  4. AI4EU [825619]

Ask authors/readers for more resources

This paper explores how to adapt the Relief algorithm to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, proposing the faster and better feature ranking ReliefE algorithm. Evaluation on 20 real-life data sets shows the utility of ReliefE for high-dimensional data sets.
Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embedding-based methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding's dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available