Dense Embeddings
How dense embeddings turn meaning into geometry: word2vec, GloVe, and contextual models, vector arithmetic, cosine similarity, and where the field is heading.
9 min readConcept
Tagged with
embeddings
Explore machine learning concepts related to embeddings. Clear explanations and practical insights.
12
Concepts Found
Concepts Related to embeddings
Pooling Strategies
How a transformer’s per-token outputs become one embedding: CLS, mean, max, last-token, and attention pooling — what each does and when to use it.
5 min readConcept
Matryoshka Embeddings
Matryoshka embeddings: nested representations enabling dimension reduction by simple truncation without model retraining for flexible retrieval.
5 min readConcept
Quantization Effects Simulator
Embedding quantization simulator: explore memory-accuracy trade-offs from float32 to int8 and binary representations for retrieval.
5 min readConcept
Vector Quantization Techniques
Master vector compression techniques from scalar to product quantization. Learn how to reduce memory usage by 10-100× while preserving search quality.
5 min readConcept
Vector Index Structures
Explore the fundamental data structures powering vector databases: trees, graphs, hash tables, and hybrid approaches for efficient similarity search.
5 min readConcept
HNSW vs IVF-PQ vs LSH: Approximate Nearest Neighbor Algorithms Compared
How HNSW, IVF-PQ, and LSH compare for approximate nearest neighbor (ANN) search — recall, latency, memory, build cost, and update characteristics — with Annoy, ScaNN, and DiskANN included for completeness.
5 min readConcept
LSH: Locality Sensitive Hashing
Explore how LSH uses probabilistic hash functions to find similar vectors in sub-linear time, perfect for streaming and high-dimensional data.
4 min readConcept
IVF-PQ: Inverted File with Product Quantization
Learn how IVF-PQ combines clustering and compression to enable billion-scale vector search with minimal memory footprint.
5 min readConcept
HNSW: Hierarchical Navigable Small World
How HNSW navigates a layered proximity graph to find nearest neighbors in logarithmic time — the default in-memory index of modern vector databases.
6 min readConcept
Representation Collapse in Self-Supervised Learning
Understanding complete, dimensional, and cluster collapse — the failure modes that every self-supervised method must prevent. Learn why collapse happens and how contrastive, asymmetric, regularization, and masking approaches solve it.
7 min readConcept
The Modality Gap in Multimodal AI
The modality gap in CLIP and vision-language models: why image and text embeddings occupy separate regions despite contrastive training.
6 min readConcept