☆ 4.7 Article

Student Network Learning via Evolutionary Knowledge Distillation

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY (2022)

Learning Student Networks via Feature Embedding

Hanting Chen et al.

Summary: Knowledge distillation optimizes a portable student network by transferring knowledge from a well-trained heavy teacher network. This article proposes a new method that propagates information through feature embedding and locality preserving loss, enabling the student network to inherit high-dimensional features from the teacher network. Experimental results show that the proposed algorithm outperforms state-of-the-art teacher-student learning methods in terms of computational and storage complexity.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2021)

添加到收藏夹

Article Computer Science, Artificial Intelligence

Knowledge Distillation: A Survey

Jianping Gou et al.

Summary: This paper provides a comprehensive survey of knowledge distillation, covering knowledge categories, training schemes, teacher-student architecture, distillation algorithms, performance comparison, and applications. It also briefly reviews challenges in knowledge distillation and discusses future research directions.

INTERNATIONAL JOURNAL OF COMPUTER VISION (2021)

添加到收藏夹

Article Engineering, Electrical & Electronic