Explore machine learning concepts related to Normalization. Clear explanations and practical insights.
BatchNorm normalizes over the batch and spatial axes; LayerNorm normalizes over the channel and spatial axes for each sample. The choice changes whether your model trains stably with batch=1, depends on batch composition at inference, and behaves consistently across train and eval.
Learn layer normalization for transformers and sequence models: how normalizing across features enables batch-independent training.
Understand internal covariate shift: why layer input distributions change during training, how it slows convergence, and how batch norm fixes it.
Learn batch normalization in deep learning: how normalizing layer inputs accelerates training, improves gradient flow, and acts as regularization.