arXiv 2024
Liger RMSNorm
Liger Kernel: Fused RMSNorm
Pin-Lun Hsu, Yun Dai, +8
Derive RMSNorm, trace fused rowwise normalization, inspect mixed-precision cast boundaries, and follow the input and shared-weight gradients in Liger Kernel.
Explore machine learning papers and reviews related to Normalization. Find insights, analysis, and implementation details.
Liger RMSNorm
Pin-Lun Hsu, Yun Dai, +8
Derive RMSNorm, trace fused rowwise normalization, inspect mixed-precision cast boundaries, and follow the input and shared-weight gradients in Liger Kernel.
BatchNorm
Sergey Ioffe, Christian Szegedy
Trace mini-batch statistics, gradients shared across samples, learned scale and shift, and frozen inference; then inspect the original Inception evidence and the limits of its causal explanation.