Maximum relevancy maximum complementary based ordered aggregation for ensemble pruning

Ensemble methods have delivered exceptional performance in various applications. However, this exceptional performance is achieved at the expense of heavy storage requirements and slower predictions. Ensemble pruning aims at reducing the complexity of this popular learning paradigm without worsening its performance. This paper presents an efficient and effective ordering-based ensemble pruning methods which ranks all the base classifiers with respect to a maximum relevancy maximum complementary (MRMC) measure. The MRMC measure evaluates the base classifier's classification ability as well as its complementariness to the ensemble, and thereby a set of accurate and complementary base classifiers can be selected. Moreover, an evaluation function that deliberately favors the candidate sub-ensembles with a better performance in classifying low margin instances has also been proposed. Experiments performed on 25 benchmark datasets demonstrate the effectiveness of our proposed method.