Explore machine learning papers and reviews related to knowledge-distillation. Find insights, analysis, and implementation details.
How DINOv2 combines DINO self-distillation with iBOT masked prediction at scale on curated data (LVD-142M), producing the strongest open-source frozen visual features across classification, segmentation, depth, and retrieval.
How self-supervised learning works without negative pairs — a predictor and momentum target network are all you need to prevent representation collapse.
How self-distillation with no labels produces Vision Transformer attention maps that automatically segment objects — without any pixel-level supervision.