Layer Normalization for Transformers
Learn layer normalization for transformers and sequence models: how normalizing across features enables batch-independent training.
Related
9 min read
Clear explanations of core machine learning concepts, from foundational ideas to advanced techniques. Understand attention mechanisms, transformers, skip connections, and more.
Learn layer normalization for transformers and sequence models: how normalizing across features enables batch-independent training.
9 min read
A fine kernel can still profile poorly: warp divergence taxes throughput ~1/N; alone on a long scoreboard wait, SM issue util can hit 0%. Interactive demos.
11 min read · intermediate
Understand C++ pointers and references with step-by-step memory diagrams: addressing, dereferencing, aliasing, array arithmetic, and object lifetime.
7 min read · intermediate
RAID storage visualized: RAID 0, 1, 5, 6, and 10 levels explained. Learn how they work, when to use them, and disk failure recovery.
18 min read
Trace top-k expert routing, normalized output weights and capacity overflow, then compare stored parameters and expert work with explicit compute baselines.
12 min read · advanced
Follow a 3D world point through camera extrinsics, depth division and pixel intrinsics, then see why one image point cannot determine its depth.
7 min read · beginner