Interactive Look: Self-Attention in Vision Transformers
Explore how self-attention enables Vision Transformers (ViT) to understand images by capturing global context, with CNN comparison.
No direct links0 refs
ML Concepts - Page 14
Clear explanations of core machine learning concepts, from foundational ideas to advanced techniques. Understand attention mechanisms, transformers, skip connections, and more.
174
Total Concepts
325
Topics
Page 14
of 29
Concepts - Page 14
Transparent Huge Pages (THP): Reducing TLB Pressure
Learn how Transparent Huge Pages (THP) reduces TLB misses by promoting 4KB to 2MB pages. Understand performance benefits and memory bloat tradeoffs.
No direct links0 refs
MHA vs GQA vs MQA: Choosing the Right Attention
Compare Multi-Head, Grouped-Query, and Multi-Query Attention mechanisms to understand their trade-offs and choose the optimal approach for your use case.
No direct links0 refs
ALiBi: Attention with Linear Biases
Learn ALiBi, the position encoding method that adds linear biases to attention scores for exceptional length extrapolation in transformers.
No direct links0 refs
Attention Sinks: Stable Streaming LLMs
Learn about attention sinks, where LLMs concentrate attention on initial tokens, and how preserving them enables streaming inference.
No direct links0 refs
Cross-Attention: Bridging Different Modalities
Understand cross-attention, the mechanism that enables transformers to align and fuse information from different sources, sequences, or modalities.
No direct links0 refs
Showing 79-84 of 174 articles