4.6 Review

A review of convolutional neural network architectures and their optimizations

相关参考文献

注意:仅列出部分参考文献,下载原文获取全部文献信息。
Proceedings Paper Computer Science, Artificial Intelligence

Mobile-Former: Bridging MobileNet and Transformer

Yinpeng Chen et al.

Summary: Mobile-Former is a parallel design of MobileNet and transformer with a two-way bridge, combining the advantages of both models for efficient computation and enhanced representation power across image classification and object detection tasks.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022) (2022)

Article Computer Science, Artificial Intelligence

Nonlinear tensor train format for deep neural network compression

Dingheng Wang et al.

Summary: This research introduces a novel nonlinear tensor train (NTT) format by studying various tensor decomposition methods, which compensates for the accuracy loss that normal tensor train (TT) cannot provide by embedding additional nonlinear activation functions in sequenced contractions and convolutions. Experimental results demonstrate that the compressed DNNs in the NTT format can maintain accuracy on multiple datasets.

NEURAL NETWORKS (2021)

Article Computer Science, Artificial Intelligence

COP: customized correlation-based Filter level pruning method for deep CNN compression

Wenxiao Wang et al.

Summary: As deep CNNs become larger, deploying them on mobile devices with limited resources becomes more challenging. Filter-level pruning is a popular method to compress deep models for mobile deployment, but it still faces issues such as high redundancy and sub-optimality.

NEUROCOMPUTING (2021)

Review Computer Science, Artificial Intelligence

A review of deep learning used in the hyperspectral image analysis for agriculture

Chunying Wang et al.

Summary: Hyperspectral imaging is a non-destructive, nonpolluting, and fast technology used for agricultural information acquisition and quality attribute detection. Deep learning techniques have significantly enhanced the performance of hyperspectral image analysis, particularly in applications such as predicting ripeness and components of agricultural products, and detecting plant diseases.

ARTIFICIAL INTELLIGENCE REVIEW (2021)

Article Computer Science, Artificial Intelligence

BlockQNN: Efficient Block-Wise Neural Network Architecture Generation

Zhao Zhong et al.

Summary: In this paper, the authors propose BlockQNN, a block-wise network generation pipeline that automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The approach outperformed hand-crafted networks on image classification tasks and showed strong generalizability across datasets.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2021)

Article Computer Science, Artificial Intelligence

QTTNet: Quantized tensor train neural networks for 3D object and video recognition

Donghyun Lee et al.

Summary: This article introduces a training framework for three-dimensional convolutional neural networks called QTTNet, which combines tensor train decomposition and data quantization to further shrink the model size and reduce memory and time costs. Experimental results demonstrate the effectiveness and competitiveness of this method in compressing 3D object and video recognition models.

NEURAL NETWORKS (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Generalizable Mixed-Precision Quantization via Attribution Rank Preservation

Ziwei Wang et al.

Summary: This paper introduces a generalizable mixed-precision quantization method, which significantly reduces search cost on large-scale datasets without performance degradation. By accurately locating network attribution across different data distributions, the method achieves a competitive accuracy-complexity trade-off.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation

Lewei Yao et al.

Summary: This paper introduces a novel knowledge distillation strategy for object detection, incorporating feature imitation and contrastive distillation techniques. The method outperforms existing detection KD techniques for both homogeneous and heterogeneous student-teacher pairs.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Full-Duplex Strategy for Video Object Segmentation

Ge-Peng Ji et al.

Summary: The paper introduces a novel framework called FSNet, which utilizes a relational cross-attention module and bidirectional purification module to achieve bidirectional message passing and improve model robustness. By performing cross-modal feature-passing before fusion and decoding, the model demonstrates robustness to various challenging scenarios in video object segmentation.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Recurrent Mask Refinement for Few-Shot Medical Image Segmentation

Hao Tang et al.

Summary: This work proposes a new framework for few-shot medical image segmentation based on prototypical networks, achieving substantial improvement over state-of-the-art methods in experiments by designing a context relation encoder and a recurrent mask refinement module.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

BN-NAS: Neural Architecture Search with Batch Normalization

Boyu Chen et al.

Summary: BN-NAS is a method for accelerating neural architecture search using Batch Normalization, which predicts subnet performance and improves training efficiency by training only BN parameters.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Generative Adversarial Registration for Improved Conditional Deformable Templates

Neel Dey et al.

Summary: Deformable templates are crucial for various medical image processing tasks, but traditional methods often produce blurry or anatomically implausible templates. This study introduces an adversarial game approach to enhance the realism of moved templates, resulting in templates with better specificity to attributes and improved sharpness and centrality, enabling more accurate population modeling and anatomical delineation for biomedical analysis.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Joint Inductive and Transductive Learning for Video Object Segmentation

Yunyao Mao et al.

Summary: This study introduces a unified framework that integrates transductive and inductive learning for accurate and robust video object segmentation by exploiting the complementarity between the two. The approach consists of two functional branches, one for aggregating spatio-temporal cues and the other for obtaining discriminative target information.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Distilling Holistic Knowledge with Graph Neural Networks

Sheng Zhou et al.

Summary: Knowledge Distillation (KD) aims to transfer knowledge from a teacher network to a student network, with individual and relational knowledge being crucial components, but existing methods often overlook the inherent correlations between them.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

AutoFormer: Searching Transformers for Visual Recognition

Minghao Chen et al.

Summary: AutoFormer is a novel one-shot architecture search framework dedicated to vision transformer search. It outperforms recent models like ViT and DeiT, achieving good accuracy on ImageNet by training a supernet and generating comparable subnets.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Convolutional Auto-Encoder with Tensor-Train Factorization

Manish Sharma et al.

Summary: Convolutional auto-encoders (CAEs) are often over-parameterized when trained with small or moderate-sized datasets, leading to decreased performance.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

ViT-YOLO:Transformer-Based YOLO for Object Detection

Zixiao Zhang et al.

Summary: The study introduces an improved backbone network, MHSA-Darknet, for object detection, utilizing multi-head self-attention and a weighted bi-directional feature pyramid network for feature fusion. Additional techniques like TTA and WBF are employed to enhance accuracy and robustness, leading to significant performance improvements over state-of-the-art detectors in the VisDrone-DET 2021 challenge.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Densely connected multidilated convolutional networks for dense prediction tasks

Naoya Takahashi et al.

Summary: This paper introduces a novel CNN architecture called D3Net, which emphasizes the importance of simultaneously modeling different resolutions using multidilated convolution with varying dilation factors in a single layer. By combining multidilated convolution with DenseNet architecture, D3Net achieves multiresolution learning with an exponentially growing receptive field in almost all layers, outperforming state-of-the-art methods in image semantic segmentation and audio source separation tasks.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Manifold Regularized Dynamic Network Pruning

Yehui Tang et al.

Summary: The proposed approach in this paper dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the method is verified on several benchmarks, showing better performance in terms of both accuracy and computational cost compared to state-of-the-art methods. This new paradigm maximally excavates redundancy in the network architecture.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

ReNAS: Relativistic Evaluation of Neural Architecture Search

Yixing Xu et al.

Summary: This study introduces a relative performance prediction method for neural architecture search, which can find better-performing architectures while saving computational costs. Experimental results show that sampling a small number of neural architectures on the NAS-Bench-101 dataset is sufficient to learn an accurate architecture performance predictor.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Learnable Companding Quantization for Accurate Low-bit Neural Networks

Kohei Yamamoto

Summary: The proposed learnable companding quantization (LCQ) method optimizes model weights and companding functions to control non-uniform quantization, along with a new weight normalization technique. Experimental results show that LCQ outperforms conventional methods in image classification and object detection tasks, narrowing the gap between quantization and full-precision models.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images

Meng Ye et al.

Summary: Our study introduces a deep learning-based unsupervised method for in vivo motion tracking on t-MRI images, which utilizes temporal information for reasonable estimations on spatiotemporal motion fields, providing a useful solution for motion tracking and image registration in dynamic medical imaging.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Cross Modal Focal Loss for RGBD Face Anti-Spoofing

Anjith George et al.

Summary: This study presents a new framework for facial presentation attack detection using RGB and depth channels together with a novel loss function. By optimizing the cross-modal focal loss function, it effectively improves the robustness of PAD systems. Extensive evaluations on publicly available datasets demonstrate the effectiveness of the proposed approach.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework

Zhizhong Huang et al.

Summary: This paper proposes a unified, multi-task framework MTLFace to handle age-invariant identity-related representation and face synthesis, by decomposing face features using an attention mechanism, removing age correlation through multi-task training and continuous domain adaption, introducing an identity conditional module for identity-level face synthesis, and achieving superior performance in experiments.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Dynamic Slimmable Network

Changlin Li et al.

Summary: The study introduces a dynamic thinning network mechanism called DS-Net, aiming to dynamically adjust the number of filters in the network at test time to achieve hardware efficiency while maintaining static storage to prevent extra burden. Dynamic inference is achieved through a double-headed dynamic gate, with a two-stage training scheme proposed for the generality of candidate architectures and fairness of the gate.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens

Zhaohui Yang et al.

Summary: This paper proposes an hourglass-inspired approach (HourNAS) for extremely fast Neural Architecture Search (NAS) by identifying vital blocks and making them the priority in the architecture search. Experimental results show that HourNAS can search for a high-performing architecture in a very short period of time.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

3D Graph Anatomy Geometry-Integrated Network for Pancreatic Mass Segmentation, Diagnosis, and Quantitative Patient Management

Tianyi Zhao et al.

Summary: The study proposes a holistic segmentation-mesh-classification network (SMCN) for distinguishing pancreatic ductal adenocarcinoma (PDAC) from nonPDAC masses, utilizing both image appearance and the 3D organ-mass geometry relationship. SMCN combines anatomical structure and semantic detection to provide patient-level diagnosis, achieving improved mass segmentation and detection accuracy compared to the baseline method, and comparable results to expert radiologists and multimodality testing for clinical management.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Complementary Relation Contrastive Distillation

Jinguo Zhu et al.

Summary: This paper introduces a novel knowledge distillation method, Complementary Relation Contrastive Distillation (CRCD), which transfers structural knowledge from teacher to student using anchor points. By maximizing mutual information between anchor-teacher and anchor-student relations, it effectively distills sample representations and inter-sample relations. Experimental results on various benchmarks demonstrate the effectiveness of the proposed CRCD.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Adaptive Binary-Ternary Quantization

Ryan Razani et al.

Summary: Neural network models are resource intensive and difficult to deploy on devices with limited resources. Low bit quantization such as binary and ternary quantization can alleviate this issue, with ternary quantization being more accurate. Mixed quantized models allow a trade-off between accuracy and memory footprint.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2021 (2021)

Proceedings Paper Computer Science, Artificial Intelligence

BCNN: A Binary CNNWith All Matrix Ops Quantized To 1 Bit Precision

Arthur J. Redfern et al.

Summary: This paper introduces a CNN with fully binary 2D convolution operations, achieving 71.24% top 1 accuracy on the ImageNet validation set. The network structure is derived from a proof showing that binary neural networks are universal function approximators, and uses implementation strategies optimized for binary operands.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2021 (2021)

Review Computer Science, Theory & Methods

Review of deep learning: concepts, CNN architectures, challenges, applications, future directions

Laith Alzubaidi et al.

Summary: Deep learning has become the gold standard in the machine learning community, widely used in various domains and capable of learning massive data. Through a comprehensive survey, a better understanding of the most important aspects of deep learning is provided.

JOURNAL OF BIG DATA (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Combining Weight Pruning and Knowledge Distillation For CNN Compression

Nima Aghli et al.

Summary: Model compression is crucial for deep neural networks, with popular methods like weight pruning not suitable for complex networks like ResNets.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2021 (2021)

Article Computer Science, Artificial Intelligence

Neural networks for facial age estimation: a survey on recent advances

Prachi Punyani et al.

ARTIFICIAL INTELLIGENCE REVIEW (2020)

Article Computer Science, Artificial Intelligence

A comprehensive survey on model compression and acceleration

Tejalal Choudhary et al.

ARTIFICIAL INTELLIGENCE REVIEW (2020)

Article Computer Science, Artificial Intelligence

'Squeeze & excite' guided few-shot segmentation of volumetric images

Abhijit Guha Roy et al.

MEDICAL IMAGE ANALYSIS (2020)

Article Computer Science, Artificial Intelligence

A survey of the recent architectures of deep convolutional neural networks

Asifullah Khan et al.

ARTIFICIAL INTELLIGENCE REVIEW (2020)

Article Computer Science, Artificial Intelligence

Block-term tensor neural networks

Jinmian Ye et al.

NEURAL NETWORKS (2020)

Proceedings Paper Computer Science, Artificial Intelligence

Rotation Consistent Margin Loss for Efficient Low-bit Face Recognition

Yudong Wu et al.

2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2020)

Article Computer Science, Artificial Intelligence

Recent advances in convolutional neural network acceleration

Qianru Zhang et al.

NEUROCOMPUTING (2019)

Article Computer Science, Hardware & Architecture

A High-Throughput and Power-Efficient FPGA Implementation of YOLO CNN for Object Detection

Duy Thanh Nguyen et al.

IEEE TRANSACTIONS ON VERY LARGE SCALE INTEGRATION (VLSI) SYSTEMS (2019)

Article Computer Science, Information Systems

A Spectral Clustering Based Filter-Level Pruning Method for Convolutional Neural Networks

Lianqiang Li et al.

IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS (2019)

Article Computer Science, Artificial Intelligence

Compressing CNN-DBLSTM models for OCR with teacher-student learning and Tucker decomposition

Haisong Ding et al.

PATTERN RECOGNITION (2019)

Proceedings Paper Computer Science, Artificial Intelligence

NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm

Zhichao Lu et al.

PROCEEDINGS OF THE 2019 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE (GECCO'19) (2019)

Proceedings Paper Computer Science, Artificial Intelligence

Learning Channel-wise Interactions for Binary Convolutional Neural Networks

Ziwei Wang et al.

2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019) (2019)

Proceedings Paper Computer Science, Hardware & Architecture

Boosting the Performance of CNN Accelerators with Dynamic Fine-Grained Channel Gating

Weizhe Hua et al.

MICRO'52: THE 52ND ANNUAL IEEE/ACM INTERNATIONAL SYMPOSIUM ON MICROARCHITECTURE (2019)

Article Computer Science, Information Systems

Pruning Blocks for CNN Compression and Acceleration via Online Ensemble Distillation

Zongyue Wang et al.

IEEE ACCESS (2019)

Article Computer Science, Artificial Intelligence

Recent advances in convolutional neural networks

Jiuxiang Gu et al.

PATTERN RECOGNITION (2018)

Proceedings Paper Computer Science, Artificial Intelligence

Two-Step Quantization for Low-bit Neural Networks

Peisong Wang et al.

2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2018)

Proceedings Paper Computer Science, Artificial Intelligence

Deep Pyramidal Residual Networks

Dongyoon Han et al.

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017) (2017)

Proceedings Paper Computer Science, Artificial Intelligence

Fixed-point Factorized Networks

Peisong Wang et al.

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017) (2017)

Article Computer Science, Hardware & Architecture

Structured Pruning of Deep Convolutional Neural Networks

Sajid Anwar et al.

ACM JOURNAL ON EMERGING TECHNOLOGIES IN COMPUTING SYSTEMS (2017)

Proceedings Paper Computer Science, Artificial Intelligence

Densely Connected Convolutional Networks

Gao Huang et al.

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017) (2017)

Proceedings Paper Computer Science, Artificial Intelligence

Channel Pruning for Accelerating Very Deep Neural Networks

Yihui He et al.

2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV) (2017)

Proceedings Paper Computer Science, Artificial Intelligence

Xception: Deep Learning with Depthwise Separable Convolutions

Francois Chollet

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017) (2017)

Article Computer Science, Artificial Intelligence

Accelerating Very Deep Convolutional Networks for Classification and Detection

Xiangyu Zhang et al.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2016)

Article Robotics

A Taxonomy of Deep Convolutional Neural Nets for Computer Vision

Suraj Srinivas et al.

FRONTIERS IN ROBOTICS AND AI (2016)

Article Computer Science, Artificial Intelligence

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

Kaiming He et al.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2015)

Proceedings Paper Computer Science, Artificial Intelligence

Fast R-CNN

Ross Girshick

2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV) (2015)

Article Computer Science, Theory & Methods

Spoofing Face Recognition With 3D Masks

Nesli Erdogmus et al.

IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY (2014)

Article Engineering, Electrical & Electronic

A Low-Power Processor With Configurable Embedded Machine-Learning Accelerators for High-Order and Adaptive Analysis of Medical-Sensor Signals

Kyong Ho Lee et al.

IEEE JOURNAL OF SOLID-STATE CIRCUITS (2013)

Article Computer Science, Artificial Intelligence

Selective Search for Object Recognition

J. R. R. Uijlings et al.

INTERNATIONAL JOURNAL OF COMPUTER VISION (2013)

Article Computer Science, Artificial Intelligence

Product Quantization for Nearest Neighbor Search

Herve Jegou et al.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2011)

Article Mathematics, Applied

TENSOR-TRAIN DECOMPOSITION

I. V. Oseledets

SIAM JOURNAL ON SCIENTIFIC COMPUTING (2011)

Article Multidisciplinary Sciences

Reducing the dimensionality of data with neural networks

G. E. Hinton et al.

SCIENCE (2006)