Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun,
Mario Lucic, and Cordelia Schmid. Vivit: A video vision
transformer. In Proceedings of the IEEE international
conference on computer vision, 2021.
Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr
Bojanowski, Florian Bordes, Pascal Vincent, Armand
Joulin, Michael Rabbat, and Nicolas Ballas. Masked
siamese networks for label-efficient learning.arXiv preprint
arXiv:2204.07141, 2022.
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bo-
janowski, Pascal Vincent, Michael Rabbat, Yann LeCun,
and Nicolas Ballas. Self-supervised learning from images
with a joint-embedding predictive architecture. InProceed-
ings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition, pages 15619–15629, 2023.
Alexei Baevski, Arun Babu, Wei-Ning Hsu, and Michael
Auli. Efficient self-supervised learning with contextualized
target representations for vision, speech and language.
arXiv preprint arXiv:2212.07525, 2022a.
Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu,
Jiatao Gu, and Michael Auli. Data2vec: A general frame-
work for self-supervised learning in speech, vision and
language. arXiv preprint arXiv:2202.03555, 2022b.
Hangbo Bao, Li Dong, and Furu Wei. Beit: Bert pre-training
of image transformers.arXiv preprint arXiv:2106.08254,
2021.
Pietro Berkes and Laurenz Wiskott. Slow feature analysis
yields a rich repertoire of complex cell properties.Journal
of vision, 5(6):9–9, 2005.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal,
Piotr Bojanowski, and Armand Joulin. Unsupervised learn-
ing of visual features by contrasting cluster assignments.
arXiv preprint arXiv:2006.09882, 2020.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jé-
gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.
Emerging properties in self-supervised vision transformers.
arXiv preprint arXiv:2104.14294, 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge-
offrey Hinton. A simple framework for contrastive learning
of visual representations.preprint arXiv:2002.05709, 2020.
Xiaokang Chen, Mingyu Ding, Xiaodi Wang, Ying Xin,
Shentong Mo, Yunhao Wang, Shumin Han, Ping Luo,
Gang Zeng, and Jingdong Wang. Context autoencoder
for self-supervised representation learning.arXiv preprint
arXiv:2202.03026, 2022.
Xinlei Chen, Saining Xie, and Kaiming He. An empirical
study of training self-supervised vision transformers.arXiv
preprint arXiv:2104.02057, 2021.
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell
Wortsman, Gabriel Ilharco, Cade Gordon, Christoph
Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Repro-
ducible scaling laws for contrastive language-image learn-
ing. InProceedings of the IEEE/CVF Conference on Com-
puter Vision and Pattern Recognition, pages 2818–2829,
2023.
Ekin Dogus Cubuk, Barret Zoph, Vijay Mane, Dandelion and-
Vasudevan, and Quoc V. Le. Autoaugment: Learning aug-
mentation policies from data. InProceedings of the IEEE
Conference on Computer Vision and Pattern Recognition,
2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk
Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa
Dehghani, Matthias Minderer, Georg Heigold, Sylvain
Gelly, et al. An image is worth 16x16 words: Trans-
formers for image recognition at scale. arXiv preprint
arXiv:2010.11929, 2020.
Christoph Feichtenhofer, Haoqi Fan, Bo Xiong, Ross Girshick,
and Kaiming He. A large-scale study on unsupervised spa-
tiotemporal representation learning. Proceedings of the
IEEE conference on computer vision and pattern recogni-
tion, 2021.
Christoph Feichtenhofer, Yanghao Li, Kaiming He, et al.
Masked autoencoders as spatiotemporal learners.Advances
in neural information processing systems, 35:35946–35958,
2022.
David J Field. What is the goal of sensory coding?Neural
computation, 6(4):559–601, 1994.
Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick
Pérez, and Matthieu Cord. Learning representations by
predicting bags of visual words. In Proceedings of the