Interactive Look: Self-Attention in Vision Transformers
Explore how self-attention enables Vision Transformers (ViT) to understand images by capturing global context, with CNN comparison.
No direct links0 refs
ML Concepts - Page 17
Clear explanations of core machine learning concepts, from foundational ideas to advanced techniques. Understand attention mechanisms, transformers, skip connections, and more.
187
Total Concepts
356
Topics
Page 17
of 32
Concepts - Page 17
Transparent Huge Pages (THP): Reducing TLB Pressure
Learn how Transparent Huge Pages (THP) reduces TLB misses by promoting 4KB to 2MB pages. Understand performance benefits and memory bloat tradeoffs.
No direct links0 refs
ALiBi: Attention with Linear Biases
Learn ALiBi, the position encoding method that adds linear biases to attention scores for exceptional length extrapolation in transformers.
No direct links0 refs
MHA vs GQA vs MQA: Choosing the Right Attention
Compare Multi-Head, Grouped-Query, and Multi-Query Attention mechanisms to understand their trade-offs and choose the optimal approach for your use case.
No direct links0 refs
Attention Sinks: Stable Streaming LLMs
Learn about attention sinks, where LLMs concentrate attention on initial tokens, and how preserving them enables streaming inference.
No direct links0 refs
Cross-Attention: Bridging Different Modalities
Understand cross-attention, the mechanism that enables transformers to align and fuse information from different sources, sequences, or modalities.
No direct links0 refs
Showing 97-102 of 187 articles