Cross-Attention: Bridging Different Modalities
Understand cross-attention, the mechanism that enables transformers to align and fuse information from different sources, sequences, or modalities.
7 min readConcept
Tagged with
Multimodal
Explore machine learning concepts related to Multimodal. Clear explanations and practical insights.
6
Concepts Found
Concepts Related to Multimodal
The Vision-Language Alignment Problem
How vision-language models align visual and text representations using contrastive learning, cross-modal attention, and CLIP-style training.
7 min readConcept
The Modality Gap in Multimodal AI
The modality gap in CLIP and vision-language models: why image and text embeddings occupy separate regions despite contrastive training.
6 min readConcept
Multimodal Scaling Laws
Discover how multimodal vision-language models like CLIP, ALIGN, and LLaVA scale with data, parameters, and compute following Chinchilla-style power laws.
7 min readConcept
Vision-Language Adapters: Efficient Fine-tuning
Master LoRA, bottleneck adapters, and prefix tuning for parameter-efficient fine-tuning of vision-language models like LLaVA with minimal compute and memory.
10 min readConcept
Vision-Language Alignment: CLIP, BLIP-2, LLaVA, Flamingo
CLIP, BLIP-2, LLaVA and Flamingo align images with text by different routes. Compare their objectives, bridges, data, and which parts train or stay frozen.
10 min readConcept