Sliding Window Attention
Sliding Window Attention for long sequences: local context windows enable O(n) complexity, used in Mistral and Longformer models.
No direct links0 refs
ML Concepts - Page 19
Clear explanations of core machine learning concepts, from foundational ideas to advanced techniques. Understand attention mechanisms, transformers, skip connections, and more.
187
Total Concepts
356
Topics
Page 19
of 32
Concepts - Page 19
Sparse Attention Patterns
Explore sparse attention mechanisms that reduce quadratic complexity to linear or sub-quadratic, enabling efficient processing of long sequences.
No direct links0 refs
SoA vs AoS: Data Layout Optimization
Master Structure of Arrays (SoA) vs Array of Structures (AoS) data layouts for optimal cache efficiency, SIMD vectorization, and GPU memory coalescing.
No direct links0 refs
Contrastive Loss for Representation Learning
Understand contrastive loss for representation learning: interactive demos of InfoNCE, triplet loss, and embedding space clustering with temperature tuning.
No direct links0 refs
Dropout Regularization
Understand dropout regularization: how randomly silencing neurons prevents overfitting, the inverted dropout trick, and when to use each dropout variant.
No direct links0 refs
Focal Loss: Focusing on Hard Examples
Learn focal loss for deep learning: down-weight easy examples, focus on hard ones. Interactive demos of gamma, alpha balancing, and RetinaNet.
No direct links0 refs
Showing 109-114 of 187 articles