Domain Adaptation for Embeddings
Domain adaptation for embeddings: transfer learning to fine-tune retrieval models across domains while preventing catastrophic forgetting.
9 min readConcept
Explore machine learning concepts related to Fine-Tuning. Clear explanations and practical insights.
Domain adaptation for embeddings: transfer learning to fine-tune retrieval models across domains while preventing catastrophic forgetting.
Trace frozen weights and trainable LoRA matrices, calculate adapter parameter counts, and compare bottleneck adapters, prefix tuning and BitFit for vision-language models.
Move a pretrained encoder to a downstream task: trace frozen features, head training, partial and full fine-tuning, learning rates, and BatchNorm state.