相关参考文献
注意：仅列出部分参考文献，下载原文获取全部文献信息。
Article
Computer Science, Artificial Intelligence
Zhenwei He et al.
Summary: In this study, a Multi-Adversarial FasterRCNN (MAF) framework is proposed to address the cross-domain object detection task. The framework introduces Hierarchical Domain Feature Alignment (HDFA) and Aggregated Proposal Feature Alignment (APFA) modules to reduce domain disparities and improve detection performance. Furthermore, a Paradigm Teacher MAF (PT-MAF) framework is proposed with knowledge distillation and DualDiscriminator HDFA (D2-HDFA) modules to enhance domain adaptability and alignment.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2023)
Article
Computer Science, Artificial Intelligence
Yifan Liu et al.
Summary: This work focuses on transferring structure information from larger networks to compact ones for dense prediction tasks in computer vision. Existing knowledge distillation strategies for dense prediction tasks do not achieve optimal performance by distilling knowledge for each pixel separately. This work proposes structured distillation schemes, including pair-wise distillation and holistic distillation, to effectively distill structured knowledge. Experiments on three dense prediction tasks validate the effectiveness of the proposed approaches. Code is available at https://git.io/StructKD.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Zhenhua Huang et al.
Summary: The study proposes a Feature Map Distillation (FMD) framework for accurate recognition of low-resolution and noisy objects in surveillance environments. The framework includes Feature Decoder Distillation (FDD) and Feature Map Consistency-enforcement (FMC) components, enabling the transfer of privileged information from a teacher network to a student network, improving the performance of the student network.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2022)
Review
Computer Science, Artificial Intelligence
Lin Wang et al.
Summary: This paper discusses the recent progress of knowledge distillation (KD) and student-teacher (S-T) learning, providing a comprehensive survey of KD methods and commonly used S-T frameworks for vision tasks. The study summarizes the working principles and effectiveness of KD, and analyzes the research status of KD in vision applications. Finally, it explores the potential developments and future directions of KD and S-T learning.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)
Article
Computer Science, Artificial Intelligence
Baosheng Yu et al.
Summary: Heatmap regression is widely used in deep learning-based semantic landmark localization. However, it often suffers from sub-pixel localization problem due to quantization error. Previous methods rely on high-resolution heatmaps, leading to a trade-off between localization accuracy and computational cost. In this paper, a simple yet effective quantization system is proposed to address this issue. Experimental results on facial landmark and human pose estimation datasets demonstrate the effectiveness of the proposed method for efficient and accurate semantic landmark localization.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)
Article
Computer Science, Artificial Intelligence
Giorgos Kordopatis-Zilos et al.
Summary: This paper proposes a Knowledge Distillation framework called Distill-and-Select (DnS) for high performance and computationally efficient content-based video retrieval in large-scale datasets. The framework trains multiple student networks with different architectures to achieve various performance and efficiency trade-offs, showcasing state-of-the-art results in video retrieval tasks. DnS demonstrates excellent trade-offs between retrieval performance, computational speed, and storage space, achieving similar results to the teacher network but with significantly faster speeds and reduced storage requirements.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Zhendong Yang et al.
Summary: This paper proposes a Focal and Global Distillation (FGD) method for knowledge distillation in object detection. By separating foreground and background and rebuilding the relationship between pixels, the proposed method improves the effectiveness of knowledge distillation. Experimental results show that using FGD can significantly improve mAP on different detectors.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Yuge Huang et al.
Summary: Knowledge distillation is a technique that uses large networks to improve the performance of compact models. Previous methods focused on fully mimicking the teacher's behavior, which may not be flexible for low-capacity models. In this study, we propose a novel evaluation-oriented knowledge distillation method that directly reduces the performance gap between the teacher and student models during training, resulting in improved performance for the student model.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Minh Hieu Phan et al.
Summary: The article introduces a new CSS framework called REMINDER and a class similarity knowledge distillation method. By distilling the knowledge of old classes that are similar to new classes, the method selectively revises old classes and enhances the learning of new classes.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Yiqing Shen et al.
Summary: The paper introduces an efficient and reliable self-distillation framework named DLB, which not only improves generalization ability, but also exhibits robustness and stability, easy implementation, and no additional modifications required. Experimental results show that the proposed method outperforms state-of-the-art approaches.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Chuanguang Yang et al.
Summary: This paper proposes a novel Cross-Image Relational KD method that focuses on transferring structured pixel-to-pixel and pixel-to-region relations among images to improve semantic segmentation performance.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Borui Zhao et al.
Summary: This paper redefines the classical KD loss into two parts, target class knowledge distillation (TCKD) and non-target class knowledge distillation (NCKD), to study logit distillation. The effects of these two parts are empirically investigated, and it is revealed that the classical KD loss suppresses the effectiveness of NCKD and limits the flexibility to balance these two parts. To address these issues, Decoupled Knowledge Distillation (DKD) is proposed, which enables more efficient and flexible roles for TCKD and NCKD.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Defang Chen et al.
Summary: Knowledge distillation is a method to compress a powerful but cumbersome teacher model into a lightweight student model without sacrificing performance. A simple knowledge distillation technique can significantly narrow down the performance gap between the teacher and student models.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Shiming Chen et al.
Summary: The key challenge of zero-shot learning is how to infer the latent semantic knowledge between visual and attribute features, in order to transfer knowledge to unseen classes. To address this challenge, we propose the Mutually Semantic Distillation Network (MSDN), which progressively distills the intrinsic semantic representations between visual and attribute features.
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
(2022)
Article
Computer Science, Artificial Intelligence
Jia Chen et al.
Summary: The paper introduces a deep object co-segmentation method based on channel and spatial attention, which combines the attention mechanism with a deep neural network to enhance common semantic information. The method utilizes a Siamese encoder and decoder structure to extract and enhance multi-level semantic features of common objects. Experiment results demonstrate competitive performance on commonly used datasets.
KNOWLEDGE-BASED SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
Yuxin Peng et al.
Summary: The study introduces the concept of lifelong learning into visual-textual cross-modal correlation modeling, proposing a visual-textual lifelong knowledge distillation (VLKD) approach. By constructing a hierarchical recurrent network, knowledge from both semantic and attention levels is leveraged across domains and modalities, supporting cross-modal retrieval in lifelong scenarios across various domains.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2021)
Article
Computer Science, Artificial Intelligence
Jingdong Wang et al.
Summary: The High-Resolution Network (HRNet) maintains high-resolution representations and exchanges information across resolutions, resulting in superior performance in various applications such as human pose estimation, semantic segmentation, and object detection.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2021)
Article
Computer Science, Artificial Intelligence
Jianping Gou et al.
Summary: This paper provides a comprehensive survey of knowledge distillation, covering knowledge categories, training schemes, teacher-student architecture, distillation algorithms, performance comparison, and applications. It also briefly reviews challenges in knowledge distillation and discusses future research directions.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2021)
Article
Computer Science, Artificial Intelligence
Tao Zhao et al.
Summary: Weakly supervised temporal action localization is a practical yet challenging task, and current methods still have limited capacity in dealing with the challenges of over-localization, joint-localization, and under-localization. The proposed astute background response strategy and self-distillation learning strategy aim to address these challenges effectively.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Ange Lou et al.
Summary: This paper introduces a real-time semantic segmentation model called CFPNet, which achieves a good balance of performance, model size, and inference speed by utilizing the Channel-wise Feature Pyramid (CFP) module, achieving 70.1% class-wise mIoU on the Cityscapes dataset.
2021 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP)
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Zhiyuan Fang et al.
Summary: In this paper, we study knowledge distillation to compress a large VL model into a small one, addressing the challenge of inconsistent visual tokens extracted from different detectors causing misalignment of hidden representations and attention distributions. Results show that our proposed distillation significantly improves the performance of small VL models on image captioning and visual question answering tasks.
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021)
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Li Liu et al.
Summary: The paper introduces a new method called "Inter-Channel Correlation for Knowledge Distillation" (ICKD), which aligns the feature space of the student network with that of the teacher network by retaining inter-channel correlation. Experimental results demonstrate that ICKD outperforms existing methods in tasks such as ImageNet classification and Pascal VOC segmentation.
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021)
(2021)
Proceedings Paper
Computer Science, Information Systems
Jianquan Li et al.
Summary: Recent AI research is focused on automatically designing deep neural network architectures through neural architecture search (NAS). This paper proposes KD-NAS, a method to accelerate NAS for natural language processing by minimizing knowledge loss from a pre-trained teacher network. Experimental results show promising performance in terms of accuracy and searching speed compared to competitors.
SIGIR '21 - PROCEEDINGS OF THE 44TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL
(2021)
Proceedings Paper
Acoustics
Chenyu You et al.
Summary: The study introduces a novel distillation framework that aims to improve the performance of student models by distilling rich knowledge from language models through a training strategy involving knowledge distillation from spoken documents and written counterparts.
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021)
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Bo Li et al.
Summary: Knowledge distillation is a popular technique for model compression and acceleration, with the aim of exploring channel correlation structure and guiding the training between teacher and student models. A novel KD loss is proposed to align the channel correlation of feature maps and verify the effectiveness of the method on two benchmark datasets.
KNOWLEDGE SCIENCE, ENGINEERING AND MANAGEMENT, PT I
(2021)
Article
Computer Science, Artificial Intelligence
Xiang Wu et al.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2020)
Article
Computer Science, Artificial Intelligence
Shiming Ge et al.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2020)
