ICLR 2022
BEiT: BERT Pre-Training of Image Transformers
Hangbo Bao, Li Dong, +2
BEiT masked image modeling explained through original paper sources, exact PyTorch code, frozen DALL-E targets, and real masked-token and classification outputs.
Self Supervised Learning
Explore machine learning papers and reviews related to Self Supervised Learning. Find insights, analysis, and implementation details.
- Tagged with
- 11 papers
Papers Related to Self Supervised Learning
TMLR 2024
DINOv2: Learning Robust Visual Features without Supervision
Maxime Oquab, Timothée Darcet, +24
How DINOv2 combines DINO self-distillation with iBOT masked prediction on curated LVD-142M data, with source-defined frozen-feature evaluations for classification, segmentation, depth, and retrieval.
CVPR 2023
I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
Mahmoud Assran, Quentin Duval, +6
Follow I-JEPA from the original paper and equations to runnable Python, then inspect real pretrained predictions of masked image features.
arXiv 2025
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Mido Assran, Adrien Bardes, +27
How V-JEPA 2 scales self-supervised video learning to 1M+ hours with masked latent prediction and 3D-RoPE, then extends to V-JEPA 2-AC for robotic planning using 62 hours of robot video with recorded actions.
NeurIPS 2020
BYOL: Bootstrap Your Own Latent
Jean-Bastien Grill, Florian Strub, +12
How self-supervised learning works without negative pairs — a predictor and momentum target network enable learning without a contrastive negative-pair loss.
ICCV 2021
DINO: Emerging Properties in Self-Supervised Vision Transformers
Mathilde Caron, Hugo Touvron, +5
How self-distillation with no labels produces Vision Transformer attention maps that automatically segment objects — without any pixel-level supervision.
CVPR 2022
MAE: Masked Autoencoders Are Scalable Vision Learners
Kaiming He, Xinlei Chen, +4
How masking 75% of image patches and reconstructing pixels creates a scalable self-supervised learner that trains ViT-H to 87.8% on ImageNet-1K with label-free pre-training and an efficient visible-only encoder.
CVPR 2020
MoCo: Momentum Contrast for Unsupervised Visual Representation Learning
Kaiming He, Haoqi Fan, +3
How a momentum-updated encoder and a dictionary queue make contrastive learning practical — large dictionaries with consistent keys, no large-batch requirement.
ICML 2020
SimCLR: A Simple Framework for Contrastive Learning
Ting Chen, Simon Kornblith, +2
How a simple framework — augmentation, shared encoder, projection head, and contrastive loss — set a new standard for self-supervised visual representation learning.
TMLR 2024
V-JEPA: Learning Video Representations by Predicting in Latent Space
Adrien Bardes, Quentin Garrido, +6
How V-JEPA learns powerful video representations by predicting masked spatiotemporal regions in embedding space rather than reconstructing pixels, achieving state-of-the-art frozen features with superior label efficiency.
ICLR 2022
VICReg: Self-Supervised Learning Without Collapse
Adrien Bardes, Jean Ponce, +1
How variance, invariance, and covariance regularization enables self-supervised representation learning without negative pairs or momentum encoders.