4.6 Article

Hardware-aware approach to deep neural network optimization

Related references

Note: Only part of the references are listed.
Article Computer Science, Artificial Intelligence

An architecture-level analysis on deep learning models for low-impact computations

Hengyi Li et al.

Summary: This paper conducts a thorough study on the inference workload of DNNs, especially focusing on the advantages of CPUs and the potential improvements in efficiency. The research provides comprehensive details for optimizing DNN efficiency at both hardware and software levels.

ARTIFICIAL INTELLIGENCE REVIEW (2023)

Article Computer Science, Artificial Intelligence

Lightweight deep neural network from scratch

Hengyi Li et al.

Summary: Deep neural networks (DNNs) are overparameterized and demanding of hardware resources, posing challenges for inference applications on resource-constrained edge devices. This study proposes a mechanism called FS-DNN for determining lightweight DNN networks from scratch, which achieves superior performance in computing consumption with competitive or even better accuracy.

APPLIED INTELLIGENCE (2023)

Article Engineering, Multidisciplinary

PINN-FORM: A new physics-informed neural network for reliability analysis with partial differential equation

Zeng Meng et al.

Summary: In this study, the physics-informed neural network (PINN) is used as a black-box solution tool for complex limit state functions (LSFs) expressed as implicit partial differential equations (PDEs) in structural reliability analysis. The proposed PINN-FORM method combines PINN with the first-order reliability method (FORM) and achieves high accuracy in predicting both the solutions of PDEs and the reliability index within a single training process.

COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING (2023)

Review Computer Science, Artificial Intelligence

A review of deep learning in dentistry

Chenxi Huang et al.

Summary: Oral diseases have significant impact on human health, but they are often unnoticed in early stages. Deep learning, as a promising field in artificial intelligence, has achieved remarkable success in various domains, particularly in dentistry. This paper aims to provide an overview of recent research on deep learning applications in dentistry, with a focus on dental imaging. Deep learning algorithms excel in difficult tasks such as image segmentation and recognition, enabling accurate identification of oral conditions and abnormalities. Integration of deep learning with other oral health data offers a holistic understanding of the relationship between oral and systemic health. However, there are still many challenges that need to be addressed.

NEUROCOMPUTING (2023)

Review Computer Science, Artificial Intelligence

Review the state-of-the-art technologies of semantic segmentation based on deep learning

Yujian Mo et al.

Summary: This paper reviews the state-of-the-art technologies of semantic segmentation based on deep learning and investigates related works on weakly-supervised, domain adaptation, multi-modal data fusion, and real-time semantic segmentation.

NEUROCOMPUTING (2022)

Review Computer Science, Artificial Intelligence

Groundwater level prediction using machine learning models: A comprehensive review

Hai Tao et al.

Summary: This review article aims to provide an overview of state-of-the-art machine learning models implemented for groundwater level (GWL) modeling. It includes a collection of 138 articles from 2008 to 2020, summarizing the details of model types, data span, time scale, input and output parameters, performance criteria, and recommendations for future research directions.

NEUROCOMPUTING (2022)

Article Computer Science, Artificial Intelligence

UFKT: Unimportant filters knowledge transfer for CNN pruning

Sarvani Ch et al.

Summary: As deep learning models have become widely used, reducing the size of the models without compromising performance is in high demand. This paper proposes a filter pruning method that minimizes information loss by utilizing knowledge from unimportant filters. Experimental results show that this method outperforms many state-of-the-art methods in terms of accuracy.

NEUROCOMPUTING (2022)

Article Computer Science, Artificial Intelligence

Compression of deep neural networks: bridging the gap between conventional-based pruning and evolutionary approach

Yidan Zhang et al.

Summary: In this paper, a novel structured pruning method called CEA-MOP is proposed to address the problem of model compression in convolutional neural networks. By utilizing conventional pruning methods for the evolutionary process, CEA-MOP achieves a delicate balance between pruning rate and model accuracy through a multiobjective optimization evolutionary model.

NEURAL COMPUTING & APPLICATIONS (2022)

Article Computer Science, Artificial Intelligence

Enhanced mechanisms of pooling and channel attention for deep learning feature maps

Hengyi Li et al.

Summary: In this article, the authors propose a fused max-average pooling (FMAPooling) operation and an improved channel attention mechanism (FMAttn) to enhance the feature representation in deep neural networks (DNNs). The effectiveness of these proposals is verified through experiments and significant improvements in accuracy are achieved.

PEERJ COMPUTER SCIENCE (2022)

Article Computer Science, Artificial Intelligence

Accelerating deep neural network filter pruning with mask-aware convolutional computations on modern CPUs

Xiu Ma et al.

Summary: This paper proposes a method called MaskACC, which accelerates the mask-based filter pruning process on modern CPU platforms and improves the computational efficiency of the pruning process.

NEUROCOMPUTING (2022)

Article Computer Science, Artificial Intelligence

FPFS: Filter-level pruning via distance weight measuring filter similarity

Wei Zhang et al.

Summary: This paper proposes a simple distance-based filter selection method called FPFS, which visualizes the similarity between filters from a global perspective and applies it to model compression. The experiments on multiple classification datasets validate the effectiveness of FPFS.

NEUROCOMPUTING (2022)

Article Computer Science, Information Systems

Automatic Group-Based Structured Pruning for Deep Convolutional Networks

Hang Wei et al.

Summary: In this paper, an automatic group-based structured pruning method with reinforcement learning named AGSPRL is proposed, which can generate pruned models with different compression rates automatically. Experimental results show that the method not only adaptively configures the number of groups, reduces accuracy by less than 1%, but also outperforms other state-of-the-art methods in terms of effectiveness and accuracy loss.

IEEE ACCESS (2022)

Proceedings Paper Computer Science, Artificial Intelligence

Revisiting Random Channel Pruning for Neural Network Compression

Yawei Li et al.

Summary: Channel pruning is an effective method to accelerate neural network inference, but lacks a fair benchmark to compare different algorithms. Recent research reveals the importance of channel configuration, giving channel pruning a new role in searching for optimal configurations. This paper proposes a random search approach to determine channel configuration and provides a new method for comparison, showing promising results compared to other pruning methods.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Article Computer Science, Artificial Intelligence

Development and deployment of a generative model-based framework for text to photorealistic image generation

Sharad Pande et al.

Summary: The task of generating photorealistic images from textual descriptions is challenging, especially for face images. AttnGAN is proposed for fine-grained text-to-face generation, showing higher quality images compared to existing methods. The approach is evaluated on the CelebA dataset using FID score and has potential applications in criminal identification and can be deployed on standalone devices like Raspberry Pi for portability.

NEUROCOMPUTING (2021)

Article Computer Science, Artificial Intelligence

Pruning and quantization for deep neural network acceleration: A survey

Tailin Liang et al.

Summary: Deep neural networks have been widely used in computer vision applications, but their complex architectures pose challenges in real-time deployment due to high computation resources and energy costs. Network compression techniques such as pruning and quantization can help overcome these challenges by reducing redundant computations. Both techniques can be used independently or together to improve the efficiency and performance of deep neural networks.

NEUROCOMPUTING (2021)

Article Computer Science, Artificial Intelligence

GenExp: Multi-objective pruning for deep neural network based on genetic algorithm

Ke Xu et al.

Summary: This study proposes an improved genetic algorithm to map the network pruning flow as a multi-objective optimization problem, finding a suitable solution that balances the DNN's model size and workload efficiently. Experiments show up to 34% further reduction in computational workload on the ResNet50 model compared to previous schemes.

NEUROCOMPUTING (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Manifold Regularized Dynamic Network Pruning

Yehui Tang et al.

Summary: The proposed approach in this paper dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the method is verified on several benchmarks, showing better performance in terms of both accuracy and computational cost compared to state-of-the-art methods. This new paradigm maximally excavates redundancy in the network architecture.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Article Computer Science, Artificial Intelligence

On hyperparameter optimization of machine learning algorithms: Theory and practice

Li Yang et al.

NEUROCOMPUTING (2020)

Article Computer Science, Artificial Intelligence

Objective Video Quality Assessment Combining Transfer Learning With CNN

Yu Zhang et al.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2020)

Article Engineering, Electrical & Electronic

Deep Constrained Low-Rank Subspace Learning for Multi-View Semi-Supervised Classification

Zhe Xue et al.

IEEE SIGNAL PROCESSING LETTERS (2019)

Proceedings Paper Computer Science, Theory & Methods

PruneTrain Fast Neural Network Training by Dynamic Sparse Model Reconfiguration

Sangkug Lym et al.

PROCEEDINGS OF SC19: THE INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS (2019)

Proceedings Paper Computer Science, Hardware & Architecture

The Dark Side of DNN Pruning

Reza Yazdani et al.

2018 ACM/IEEE 45TH ANNUAL INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE (ISCA) (2018)

Proceedings Paper Computer Science, Artificial Intelligence

Densely Connected Convolutional Networks

Gao Huang et al.

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017) (2017)

Article Automation & Control Systems

Robust nonlinear multivariable tracking control design to a manipulator with unknown uncertainties using operator-based robust right coprime factorization

Aihui Wang et al.

TRANSACTIONS OF THE INSTITUTE OF MEASUREMENT AND CONTROL (2013)

Article Automation & Control Systems

Operator-based robust non-linear control for gantry crane system with soft measurement of swing angle

Shengjun Wen et al.

INTERNATIONAL JOURNAL OF MODELLING IDENTIFICATION AND CONTROL (2012)

Article Computer Science, Software Engineering

Anatomy of high-performance matrix multiplication

Kazushige Goto et al.

ACM TRANSACTIONS ON MATHEMATICAL SOFTWARE (2008)

Article Computer Science, Hardware & Architecture

A more realistic thinning scheme for call admission control in multimedia wireless networks

Xian Wang et al.

IEEE TRANSACTIONS ON COMPUTERS (2008)

Article Computer Science, Interdisciplinary Applications

Neuro-genetic design optimization framework to support the integrated robust design optimization process in CE

Nursel Oeztuerk et al.

CONCURRENT ENGINEERING-RESEARCH AND APPLICATIONS (2006)

Article Computer Science, Interdisciplinary Applications

Integrated optimal topology design and shape optimization using neural networks

AR Yildiz et al.

STRUCTURAL AND MULTIDISCIPLINARY OPTIMIZATION (2003)