CLS Token in Vision Transformers
Learn how the CLS token acts as a global information aggregator in Vision Transformers, enabling whole-image classification through attention mechanisms.
No direct links0 refs
ML Concepts - Page 20
Clear explanations of core machine learning concepts, from foundational ideas to advanced techniques. Understand attention mechanisms, transformers, skip connections, and more.
190
Total Concepts
358
Topics
Page 20
of 32
Concepts - Page 20
Hierarchical Attention in Vision Transformers
How hierarchical (windowed, multi-scale) attention — pioneered by Swin Transformer — cuts the quadratic cost of self-attention to near-linear for high-resolution vision.
No direct links0 refs
Multi-Head Attention
How multi-head attention runs scaled dot-product attention in parallel across several representation subspaces to build context-aware token embeddings.
No direct links0 refs
Positional Embeddings in Vision Transformers
Explore how positional embeddings enable Vision Transformers (ViT) to process sequential data by encoding relative positions.
No direct links0 refs
Self-Attention in Vision Transformers
Explore how self-attention enables Vision Transformers (ViT) to understand images by capturing global context, with a CNN comparison.
No direct links0 refs
Contrastive Loss for Representation Learning
Understand contrastive loss for representation learning: interactive demos of InfoNCE, triplet loss, and embedding space clustering with temperature tuning.
No direct links0 refs
Showing 115-120 of 190 articles