Cross-Entropy Loss: The Foundation of Classification
Learn cross-entropy loss for deep learning classification. Understand probability distributions, gradients, and maximum likelihood estimation.
9 min readConcept
Tagged with
training
Explore machine learning concepts related to training. Clear explanations and practical insights.
11
Concepts Found
Concepts Related to training
VAE Latent Space: Understanding Variational Autoencoders
Explore VAE latent space in deep learning. Learn variational autoencoder encoding, decoding, interpolation, and the reparameterization trick.
6 min readConcept
He/Kaiming Initialization
Learn He (Kaiming) initialization for ReLU neural networks. This weight initialization technique prevents vanishing gradients in deep learning models.
9 min readConcept
Xavier/Glorot Initialization
Understand Xavier (Glorot) initialization, the weight initialization technique that maintains signal variance across layers for stable deep network training.
7 min readConcept
Dropout Regularization
Learn dropout regularization to prevent overfitting in neural networks by randomly deactivating neurons during training.
11 min readConcept
Gradient Flow in Deep Networks
Learn how gradients propagate through deep neural networks during backpropagation. Understand vanishing and exploding gradient problems in deep learning.
8 min readConcept
DataParallel vs DistributedDataParallel
Compare PyTorch DataParallel vs DistributedDataParallel for multi-GPU training. Learn GIL limitations, NCCL AllReduce, and DDP best practices.
6 min readConcept
Layer Normalization
Understanding layer normalization technique that normalizes inputs across features, making it ideal for sequence models and transformers.
5 min readConcept
Internal Covariate Shift
Understand internal covariate shift in deep learning - the distribution shift problem in neural networks that batch normalization addresses.
7 min readConcept
Batch Normalization
Understanding batch normalization technique that normalizes inputs to accelerate training and improve neural network performance.
5 min readConcept
Skip Connections
Learn skip connections and residual blocks in deep learning. Understand how ResNet architecture enables training of very deep neural networks.
4 min readConcept