Target Encoders, EMA, and Stop-Gradient
Follow how a learned target encoder supplies feature targets, how stop-gradient separates backward paths, and how exponential moving averages update its parameters.
6 min readConcept
Explore machine learning concepts related to Gradient Flow. Clear explanations and practical insights.
Follow how a learned target encoder supplies feature targets, how stop-gradient separates backward paths, and how exponential moving averages update its parameters.
Move a pretrained encoder to a downstream task: trace frozen features, head training, partial and full fine-tuning, learning rates, and BatchNorm state.