Related references
Note: Only part of the references are listed.
Article
Automation & Control Systems
Majid Sepahvand et al.
Summary: This paper proposes a feature-based knowledge distillation method that uses Tucker decomposition and an adaptor function to transfer the knowledge of the complex feature maps from a teacher model to a student model, reducing the complexity of the student model while maintaining recognition rate stability.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2023)
Review
Chemistry, Analytical
Shibo Zhang et al.
Summary: The development of mobile and wearable devices has enabled various applications that measure and improve our daily lives, such as activity tracking, wellness monitoring, and human-computer interaction. Many of these applications rely on low-power sensors in these devices to perform human activity recognition (HAR). In recent years, deep learning has made significant advancements in HAR on mobile and wearable devices. This paper categorizes and summarizes existing work on deep learning methods for wearables-based HAR, and provides an analysis of the current advancements, developing trends, and major challenges. The paper also presents cutting-edge frontiers and future directions for deep learning-based HAR.
Article
Engineering, Biomedical
Bing Leng et al.
Summary: This study proposes a deep learning framework using knowledge distillation method for leukocyte classification. Large models are first pre-trained on a mixed leukocyte dataset using either convolutional neural networks or Transformer structure. Then, the dark knowledge of the pre-trained models is extracted through knowledge distillation, and small models are trained. The best performing small model is selected as the final prediction model, achieving high accuracy on a mixed dataset. The proposed framework also outperforms other methods on the enhanced BCCD dataset.
BIOMEDICAL SIGNAL PROCESSING AND CONTROL
(2022)
Article
Biology
Majid Sepahvand et al.
Summary: This paper proposes a method based on knowledge distillation to overcome the dissociation between MI classification and MD classification in histopathological images of breast cancer. The experimental results show that the proposed method achieves high accuracy rates at different magnification factors.
COMPUTERS IN BIOLOGY AND MEDICINE
(2022)
Article
Computer Science, Artificial Intelligence
Zhendong Wang et al.
Summary: This paper proposes a knowledge distillation model based on Triplet Convolution Neural Networks to improve the speed and accuracy of anomaly detection in industrial CPS, while reducing the complexity of the model. Experimental results demonstrate significant advantages of the proposed model on benchmark datasets.
EXPERT SYSTEMS WITH APPLICATIONS
(2022)
Article
Computer Science, Artificial Intelligence
Hongyuan Chen et al.
Summary: This paper proposes a Super-Resolution guided Knowledge Distillation (SRKD) framework to address the challenge of low-resolution image classification. By enhancing the features of low-resolution images and minimizing the difference between the features of high-resolution images and super-resolution images, the proposed method achieves significant improvement in experimental results.
PATTERN RECOGNITION LETTERS
(2022)
Article
Sport Sciences
Eni Hysenllari et al.
Summary: This article discusses the quality of activity recognition obtained using convolutional neural network on acceleration and angular velocity data recorded from different sensor locations. Results show that the accuracy and Cohen's kappa of the network vary depending on the sensor location, with thigh sensor showing the highest levels. The study concludes that 2D-CNN performs excellently in human activity recognition, enabling real-time activity recognition even in small devices.
GERMAN JOURNAL OF EXERCISE AND SPORT RESEARCH
(2022)
Article
Computer Science, Artificial Intelligence
Majid Sepahvand et al.
Summary: This paper studies feature-based knowledge distillation and proposes a method to improve student model performance and enhance knowledge comprehension by decomposing and distilling the knowledge from the inner layers of a teacher model to the inner layers of a student model.
EXPERT SYSTEMS WITH APPLICATIONS
(2022)
Article
Computer Science, Artificial Intelligence
Wenbin Gao et al.
Summary: The paper introduces a new dual attention method called DanHAR, which combines channel and temporal attention on residual networks to enhance feature representation for sensor-based HAR tasks. Experimental results on multiple datasets demonstrate the central importance of dual attention mechanism in activity recognition tasks, showing performance improvement over other algorithms. Visualizing analysis confirms the ability of the proposed attention mechanism to capture spatial-temporal dependencies in multimodal sensing data.
APPLIED SOFT COMPUTING
(2021)
Article
Computer Science, Artificial Intelligence
Majid Sepahvand et al.
Summary: The manuscript introduces a novel method for human motion detection based on inertial sensors and a new ensemble learning approach through genetic programing. The method utilizes spatial information of human motion for feature extraction and enhances classifier optimization through genetic programing.
EXPERT SYSTEMS WITH APPLICATIONS
(2021)
Article
Computer Science, Artificial Intelligence
Cheng Dai et al.
Summary: Knowledge distillation (KD) has been widely studied in recent deep learning research, but its effectiveness is compromised when there is a significant gap in strength between the teacher and student networks. To address this issue, the TDKD method is proposed to transfer complex mappings from cumbersome models to simpler ones by reducing capacity variance. This approach has been shown to effectively improve distillation performance and reactivate previously ineffective KD methods.
APPLIED SOFT COMPUTING
(2021)
Article
Computer Science, Theory & Methods
Nidhi Dua et al.
Summary: Human Activity Recognition (HAR), an important research area, has seen a surge in interest due to the development of a deep neural network model that automates feature extraction and activity classification, achieving high accuracy without requiring manual feature extraction.
Article
Chemistry, Analytical
Sakorn Mekruksavanich et al.
Summary: Human Activity Recognition (HAR) using inertial motion data has been increasingly utilized in various applications due to the acceleration of building intelligent environments and systems. Deep learning methods are found to be more effective than traditional machine learning techniques in feature extraction from raw sensor data. A generic HAR framework based on LSTM networks for smartphone sensor data is proposed in this study, with experiments showing improved recognition performance using a 4-layer CNN-LSTM network.
Article
Computer Science, Artificial Intelligence
Jianping Gou et al.
Summary: This paper provides a comprehensive survey of knowledge distillation, covering knowledge categories, training schemes, teacher-student architecture, distillation algorithms, performance comparison, and applications. It also briefly reviews challenges in knowledge distillation and discusses future research directions.
INTERNATIONAL JOURNAL OF COMPUTER VISION
(2021)
Article
Biology
Marzia Cescon et al.
Summary: This paper presents methods for estimating physical activity and sedentary behavior from three-axis accelerometer data collected using a wrist-worn device on adults with type 1 diabetes in free-living conditions. The methods are able to accurately detect, grade and classify activity levels and specific behaviors, with potential implications for automated glucose control systems in diabetes management.
COMPUTERS IN BIOLOGY AND MEDICINE
(2021)
Article
Agriculture, Multidisciplinary
Anniek Eerdekens et al.
COMPUTERS AND ELECTRONICS IN AGRICULTURE
(2020)
Article
Computer Science, Artificial Intelligence
Zhen Qin et al.
INFORMATION FUSION
(2020)
Article
Computer Science, Artificial Intelligence
L. Minh Dang et al.
PATTERN RECOGNITION
(2020)
Article
Chemistry, Analytical
Kyungkoo Jun et al.
Article
Chemistry, Analytical
Esther Fridriksdottir et al.
Article
Computer Science, Artificial Intelligence
Yuang Liu et al.
Article
Computer Science, Information Systems
Yue Hu et al.
Article
Computer Science, Artificial Intelligence
Zhisheng Zhong et al.
Review
Computer Science, Artificial Intelligence
Yan Wang et al.
EXPERT SYSTEMS WITH APPLICATIONS
(2019)
Article
Engineering, Multidisciplinary
Lin Chen et al.
Article
Computer Science, Artificial Intelligence
Andrey Ignatov
APPLIED SOFT COMPUTING
(2018)
