Сравнение методов

	Самообучающийся Vision Transformer	Self-supervised convolutional neural network
Область	Глубокое обучение	Глубокое обучение
Семейство	Machine learning	Machine learning
Год появления	2021–2022	2018–2020
Автор метода	Caron et al. (DINO); He et al. (MAE)	LeCun, Y. (CNN backbone); Chen et al. and He et al. (self-supervised visual frameworks)
Тип	Self-supervised pre-training for vision transformers	Self-supervised deep learning
Основополагающий источник	Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021). Emerging Properties in Self-Supervised Vision Transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 9650–9660.	Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), PMLR 119, 1597–1607.
Другие названия	SSL-ViT, self-supervised ViT, unsupervised ViT pre-training, vision transformer self-supervised pre-training	Self-supervised CNN, SSL-CNN, contrastive CNN, pretext-task CNN
Связанные	4	5
Сводка	Self-supervised Vision Transformer (SSL-ViT) applies self-supervised pre-training objectives — such as masked patch prediction (MAE) or self-distillation with no labels (DINO) — to the Vision Transformer architecture, enabling powerful visual representations to be learned from large unlabeled image corpora before any task-specific fine-tuning.	A self-supervised convolutional neural network (CNN) learns powerful visual representations from unlabeled images by solving pretext tasks — such as contrastive instance discrimination or masked-patch prediction — and then fine-tunes on a small labeled set. This approach dramatically reduces dependence on large annotated datasets while preserving the spatial feature-extraction strengths of convolutional architectures.