Emergent Abilities in Large Language Models
Explore emergent abilities in large language models: sudden capabilities at scale thresholds, phase transitions, and the mirage debate.
8 min readConcept
Tagged with
LLMs
Explore machine learning concepts related to LLMs. Clear explanations and practical insights.
10
Concepts Found
Concepts Related to LLMs
Context Windows: The Memory Limits of LLMs
Interactive visualization of LLM context windows - sliding windows, expanding contexts, and attention patterns that define model memory limits.
6 min readConcept
Prompt Engineering for LLMs
Master prompt engineering for large language models: from basic composition to Chain-of-Thought, few-shot, and advanced techniques.
7 min readConcept
Flash Attention: IO-Aware Exact Attention
Interactive Flash Attention visualization - the IO-aware algorithm achieving memory-efficient exact attention through tiling and kernel fusion.
8 min readConcept
Prompt Influence Flow Through Transformer Layers
Deep dive into how different prompt components influence model behavior across transformer layers, from surface patterns to abstract reasoning.
6 min readConcept
KV Cache: The Secret to Fast LLM Inference
Interactive KV cache visualization - how key-value caching in LLM transformers enables fast text generation without quadratic recomputation.
6 min readConcept
Neural Scaling Laws Explained
Explore neural scaling laws in deep learning: power law relationships between model size, data, and compute that predict AI performance.
8 min readConcept
Tokenization: Converting Text to Numbers
Interactive exploration of tokenization methods in LLMs - BPE, SentencePiece, and WordPiece. Understand how text becomes tokens that models can process.
6 min readConcept
Speculative Decoding: Draft, Then Verify
A small draft model proposes tokens, the target model checks them all in one pass, and a rejection rule keeps the output distribution exactly the target's.
10 min readConcept
Vision-Language Alignment: CLIP, BLIP-2, LLaVA, Flamingo
CLIP, BLIP-2, LLaVA and Flamingo align images with text by different routes. Compare their objectives, bridges, data, and which parts train or stay frozen.
10 min readConcept