Skip to main content

Deep Learning

Neural network fundamentals: normalization, convolutions, losses, graph networks, and training dynamics.

  • DL
  • 37 concepts
Browse all 37 concepts

Training dynamics

Follow gradients, choose initial weights, and keep training stable.

Starting knowledge: Neural-network layers, derivatives and a basic training loop

Select a concept. Explore its connections, or open the full explanation.

Previewing Gradient flow.

Signal and initialization

Follow the chain rule, then compare how Xavier and He choose an initial scale for different activations.

  1. Signal and initialization

    Gradient flow

    Explain how repeated derivatives can shrink or amplify gradients through a network.

    Read concept

    Background: connections 1 to 1 of 1.

Normalization

Inspect batch and feature statistics, compare their uses, then optionally examine the debate about internal covariate shift.

Residual paths

Use the gradient-flow connection to understand the identity path. The illustrated ResNet block also uses convolutions and BatchNorm.

Parameter updates

Use gradients to update parameters, then separate momentum, look-ahead and adaptive step sizes.

Regularization

Inspect random masking and train/eval behavior as a way to address overfitting.

Reuse trained parameters

Choose which pretrained parameters a new task can update, and separate that choice from gradients and normalization state. This branch applies to supervised and self-supervised encoders.

Lines show suggested reading within a branch. Connections offer optional ways to explore; they are not a checklist of prerequisites.

All Deep Learning Concepts

Mastodon