CLS Token in Vision Transformers
Trace how a learned CLS row joins image patches, gathers evidence through self-attention, and becomes the image-level classification readout.
Related
7 min read · beginner
Clear explanations of core machine learning concepts, from foundational ideas to advanced techniques. Understand attention mechanisms, transformers, skip connections, and more.
Trace how a learned CLS row joins image patches, gathers evidence through self-attention, and becomes the image-level classification readout.
7 min read · beginner
Learning where to fuse multi-scale features with per-pixel, per-level fusion weights. ASFF challenges FPN's uniform fusion assumption.
15 min read · intermediate
How the silhouette score measures clustering quality for every individual point — comparing intra-cluster cohesion to nearest-cluster separation, with per-point diagnostics and explicit distance-metric assumptions.
8 min read · intermediate
How sparse retrieval (BM25/TF-IDF), dense retrieval (BERT-style embeddings), and hybrid systems that combine both compare on recall, semantic understanding, computational cost, and operational complexity for modern search.
12 min read · beginner
How NVIDIA Tensor Cores accelerate GEMM: warp-level MMA, precision dials (TF32/FP16/BF16/INT8/FP8), shape alignment and padding cost, AMP vs model.half(), and when the speedup actually shows up.
16 min read · intermediate
Follow PyTorch batches through worker queues, collation, ordered delivery and GPU transfer, then measure loading throughput and budget process memory.
22 min read · intermediate