Cross-Attention: Bridging Different Modalities
Trace a target query through source keys, attention weights, and a weighted value mix. Understand cross-attention origins, shapes, masks, and applications.
7 min readConcept
Tagged with
Multimodal
Explore machine learning concepts related to Multimodal. Clear explanations and practical insights.
6
Concepts Found
Concepts Related to Multimodal
The Vision-Language Alignment Problem
Trace one image-caption pair through cosine scores, contrastive positives and negatives, a trainable projection, and retrieval evaluation.
8 min readConcept
The Modality Gap in Multimodal AI
Measure the centroid gap between normalized image and text embeddings, then check why a smaller gap alone does not guarantee correct retrieval.
6 min readConcept
Multimodal Scaling Laws
Explore a fixed compute budget with Chinchilla’s text-only loss fit, then see why multimodal scaling needs its own data, objectives, and evaluation metrics.
7 min readConcept
Vision-Language Adapters: Efficient Fine-tuning
Trace frozen weights and trainable LoRA matrices, calculate adapter parameter counts, and compare bottleneck adapters, prefix tuning and BitFit for vision-language models.
12 min readConcept
Vision-Language Alignment: CLIP, BLIP-2, LLaVA, Flamingo
CLIP, BLIP-2, LLaVA and Flamingo align images with text by different routes. Compare their objectives, bridges, data, and which parts train or stay frozen.
10 min readConcept