Flash Attention: IO-Aware Exact Attention
Interactive Flash Attention visualization - the IO-aware algorithm achieving memory-efficient exact attention through tiling and kernel fusion.
Related
8 min read · advanced
ML Concepts - Page 22
Clear explanations of core machine learning concepts, from foundational ideas to advanced techniques. Understand attention mechanisms, transformers, skip connections, and more.
194
Total Concepts
341
Topics
Page 22
of 33
Concepts - Page 22
Browse by categoryPrompt Influence Flow Through Transformer Layers
Deep dive into how different prompt components influence model behavior across transformer layers, from surface patterns to abstract reasoning.
Related
6 min read
NVIDIA vs AMD for Deep Learning: CUDA vs ROCm and the Datacenter Accelerators
NVIDIA vs AMD for deep learning compared at both layers: the CUDA vs ROCm software moat, the microarchitecture (warp vs wavefront, SM vs CU, Tensor vs Matrix Cores), and the datacenter accelerators (H100/H200/B200 vs MI300X/MI325X).
Related
10 min read · intermediate
C++ Linking Overview
How C++ object files are linked into executables. Learn symbol resolution, static vs dynamic linking, and linker optimization.
Related
5 min read
XFS: High-Performance Parallel Filesystem
XFS internals end-to-end: allocation groups for lock-free parallel metadata, B+ trees instead of bitmaps, extent-based allocation that scales to terabytes, and delayed allocation that turns scattered writes into contiguous extents.
Related
10 min read
KV Cache: The Secret to Fast LLM Inference
Interactive KV cache visualization - how key-value caching in LLM transformers enables fast text generation without quadratic recomputation.
Related
6 min read · intermediate
Showing 127-132 of 194 concepts