Quantformer: Learning Extremely Low-Precision Vision Transformers

Article Computer Science, Artificial Intelligence

Learning Deep Binary Descriptors via Bitwise Interaction Mining

Ziwei Wang et al.

Summary: In this paper, we propose a GraphBit method for learning unsupervised deep binary descriptors to efficiently represent images. The method reduces the uncertainty of binary codes by maximizing the mutual information with input and related bits, allowing reliable binarization of ambiguous bits. Additionally, a differentiable search method called GraphBit+ is introduced to mine bitwise interaction in continuous space, reducing the computational cost of reinforcement learning. To address the issue of inaccurate instructions from fixed bitwise interaction, the unsupervised binary descriptor learning method D-GraphBit is proposed, which utilizes a graph convolutional network to reason the optimal bitwise interaction for each input sample.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2023)

添加到收藏夹

Article Computer Science, Artificial Intelligence

Learning Efficient Binarized Object Detectors With Information Compression

Ziwei Wang et al.

Summary: In this paper, a binarized neural network learning method (BiDet) is proposed for efficient object detection. BiDet utilizes the representational capacity of binary neural networks by redundancy removal, which enhances detection precision and reduces false positives.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2022)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search

Han Xiao et al.

Summary: In this paper, a Shapley value based method, Shapley-NAS, is proposed to evaluate the operation contribution for neural architecture search. By evaluating the direct influence of operations on validation accuracy, the optimal architectures with significant contributions to the tasks are derived through optimizing the supernet weights and updating the architecture parameters.

2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) (2022)

添加到收藏夹

Article Computer Science, Artificial Intelligence

Learning Channel-Wise Interactions for Binary Convolutional Neural Networks

Ziwei Wang et al.

Summary: This paper proposes a channel-wise interaction based binary convolutional neural networks (CI-BCNN) approach for efficient inference. By using reinforcement learning to mine channel-wise interactions, correct inconsistent signs, and alleviate noise in channel-wise priors, the proposed approach improves inference efficiency.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2021)

添加到收藏夹

Article Computer Science, Artificial Intelligence

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

Zhaowei Cai et al.

Summary: In object detection, the commonly used IoU threshold of 0.5 can lead to noisy detections, and performance may degrade for larger thresholds. The Cascade R-CNN architecture addresses this issue by training detectors sequentially with increasing IoU thresholds and eliminating quality mismatches at inference, resulting in state-of-the-art performance and significant improvement in high-quality detection across various datasets. The model is also generalized to instance segmentation, achieving nontrivial improvements over existing methods.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2021)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

Generalizable Mixed-Precision Quantization via Attribution Rank Preservation

Ziwei Wang et al.

Summary: This paper introduces a generalizable mixed-precision quantization method, which significantly reduces search cost on large-scale datasets without performance degradation. By accurately locating network attribution across different data distributions, the method achieves a competitive accuracy-complexity trade-off.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization

Yi Guo et al.

Summary: Model compression techniques are gaining attention for obtaining efficient AI models, with channel pruning being an important strategy. Previous methods have limitations, but the GDP algorithm proposed in this study shows promising results in improving model performance.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

GLiT: Neural Architecture Search for Global and Local Image Transformer

Boyu Chen et al.

Summary: The paper introduces a new Neural Architecture Search (NAS) method to find a better transformer architecture for image recognition. By incorporating a locality module and new search algorithms, the method allows for a trade-off between global and local information, as well as optimizing low-level design choices in each module. Through extensive experiments on the ImageNet dataset, the method demonstrates the ability to find more efficient and discriminative transformer variants compared to existing models like ResNet101 and ViT.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer

Zhihe Lu et al.

Summary: A few-shot semantic segmentation model typically consists of a CNN encoder, a CNN decoder, and a simple classifier. This study proposes to simplify the meta-learning task by focusing solely on the classifier while leaving the encoder and decoder to pre-training. Introducing a Classifier Weight Transformer (CWT) for classifier meta-learning, the method outperforms existing alternatives on standard benchmarks.

2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021) (2021)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence

Manifold Regularized Dynamic Network Pruning

Yehui Tang et al.

Summary: The proposed approach in this paper dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the method is verified on several benchmarks, showing better performance in terms of both accuracy and computational cost compared to state-of-the-art methods. This new paradigm maximally excavates redundancy in the network architecture.

2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 (2021)

添加到收藏夹

Proceedings Paper Computer Science, Artificial Intelligence