☆ 4.5 Article

Exploration of block-wise dynamic sparseness

PATTERN RECOGNITION LETTERS (2021)

Improving knowledge distillation using unified ensembles of specialized teachers

Adamantios Zaras et al.

Summary: This study introduces a method to efficiently transfer knowledge to smaller models by using multiple highly specialized teachers and diversifying the ensemble to overcome limitations of the distillation process and achieve high distillation efficiency. The proposed method demonstrates improved distillation performance on three different image datasets, even outperforming powerful state-of-the-art ensemble-based distillation methods.

PATTERN RECOGNITION LETTERS (2021)

添加到收藏夹

Article Computer Science, Artificial Intelligence