Patch Slimming for Efficient Vision Transformers

Summary: The proposed Hierarchical Visual Transformer (HVT) progressively pools visual tokens to reduce sequence length and computational cost, outperforming competitive baselines on ImageNet and CIFAR-100 datasets without introducing extra complexity. HVT enhances model capacity by scaling dimensions and showcasing improved scalability in image classification tasks.