Dense Embeddings
How dense embeddings turn meaning into geometry: word2vec, GloVe, and contextual models, vector arithmetic, cosine similarity, and where the field is heading.
Related
8 min read · beginner
Embeddings & Retrieval
Dense and sparse embeddings, quantization, and vector search for semantic retrieval.
- IR
- 19 concepts
All Embeddings & Retrieval Concepts
Sparse vs Dense vs Hybrid Retrieval: BM25, BERT, and Reranking Compared
How sparse retrieval (BM25/TF-IDF), dense retrieval (BERT-style embeddings), and hybrid systems that combine both compare on recall, semantic understanding, computational cost, and operational complexity for modern search.
Related
12 min read · beginner
BM25 Algorithm for Text Retrieval
Master the BM25 algorithm, the probabilistic ranking function powering Elasticsearch and Lucene for keyword-based document retrieval and search systems.
Related
12 min read · beginner
Pooling Strategies
How a transformer’s per-token outputs become one embedding: CLS, mean, max, last-token, and attention pooling — what each does and when to use it.
Related
9 min read · intermediate
Contrastive Learning
Master contrastive learning for vector embeddings: how InfoNCE loss and self-supervised techniques train models to create high-quality semantic representations.
Related
10 min read · advanced
Matryoshka Embeddings
Matryoshka embeddings: nested representations enabling dimension reduction by simple truncation without model retraining for flexible retrieval.
Related
10 min read · intermediate
Domain Adaptation for Embeddings
Domain adaptation for embeddings: transfer learning to fine-tune retrieval models across domains while preventing catastrophic forgetting.
Related
9 min read · intermediate
Cross-Lingual Alignment
Learn cross-lingual embedding alignment techniques like VecMap and MUSE for multilingual vector retrieval and zero-shot language transfer in search systems.
Related
8 min read · advanced
Cross-Encoder vs Bi-Encoder
Understand the fundamental differences between independent and joint encoding architectures for neural retrieval systems.
Related
10 min read · intermediate
Multi-Vector Late Interaction
Explore ColBERT and other multi-vector retrieval models that use fine-grained token-level matching for superior search quality.
Related
12 min read · advanced
Hybrid Retrieval Systems
Build hybrid retrieval systems combining BM25 sparse search with dense vector embeddings using reciprocal rank fusion for superior semantic search performance.
Related
10 min read · intermediate
Quantization Effects Simulator
Embedding quantization simulator: explore memory-accuracy trade-offs from float32 to int8 and binary representations for retrieval.
Related
10 min read · intermediate
Vector Quantization Techniques
Master vector compression techniques from scalar to product quantization. Learn how to reduce memory usage by 10-100× while preserving search quality.
Related
12 min read · intermediate
Binary Embeddings for Fast Search
Learn how binary embeddings use 1-bit quantization for ultra-compact vector representations, enabling billion-scale similarity search with 32x memory reduction.
Related
8 min read · intermediate
Vector Index Structures
Explore the fundamental data structures powering vector databases: trees, graphs, hash tables, and hybrid approaches for efficient similarity search.
Related
14 min read · intermediate
HNSW vs IVF-PQ vs LSH: Approximate Nearest Neighbor Algorithms Compared
How HNSW, IVF-PQ, and LSH compare for approximate nearest neighbor (ANN) search — recall, latency, memory, build cost, and update characteristics — with Annoy, ScaNN, and DiskANN included for completeness.
Related
15 min read · intermediate
LSH: Locality Sensitive Hashing
Explore how LSH uses probabilistic hash functions to find similar vectors in sub-linear time, perfect for streaming and high-dimensional data.
Related
10 min read · intermediate
IVF-PQ: Inverted File with Product Quantization
Learn how IVF-PQ combines clustering and compression to enable billion-scale vector search with minimal memory footprint.
Related
12 min read · advanced
HNSW: Hierarchical Navigable Small World
How HNSW navigates a layered proximity graph to find nearest neighbors in logarithmic time — the default in-memory index of modern vector databases.
Related
9 min read · advanced