Journal
2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022)
Volume -, Issue -, Pages 16549-16558Publisher
IEEE COMPUTER SOC
DOI: 10.1109/CVPR52688.2022.01607
Keywords
-
Ask authors/readers for more resources
This work studies the importance of asymmetry in visual representation learning and finds that keeping a relatively lower variance in target than source benefits learning. The improvements from asymmetric designs generalize well to longer training schedules, multiple other frameworks, and newer backbones. Finally, the combined effect of several asymmetric designs achieves state-of-the-art accuracy on ImageNet linear probing.
Many recent self-supervised frameworks for visual representation learning are based on certain forms of Siamese networks. Such networks are conceptually symmetric with two parallel encoders, but often practically asymmetric as numerous mechanisms are devised to break the symmetry. In this work, we conduct a formal study on the importance of asymmetry by explicitly distinguishing the two encoders within the network - one produces source encodings and the other targets. Our key insight is keeping a relatively lower variance in target than source generally benefits learning. This is empirically justified by our results from five case studies covering different variance-oriented designs, and is aligned with our preliminary theoretical analysis on the baseline. Moreover, we find the improvements from asymmetric designs generalize well to longer training schedules, multiple other frameworks and newer backbones. Finally, the combined effect of several asymmetric designs achieves a state-of-the-art accuracy on ImageNet linear probing and competitive results on downstream transfer. We hope our exploration will inspire more research in exploiting asymmetry for Siamese representation learning.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available