Emergent Abilities in Large Language Models
Explore emergent abilities in large language models: sudden capabilities at scale thresholds, phase transitions, and the mirage debate.
Explore machine learning concepts related to LLMs. Clear explanations and practical insights.
Explore emergent abilities in large language models: sudden capabilities at scale thresholds, phase transitions, and the mirage debate.
See how context capacity, attention masks, and KV caches differ. Move a planted fact through a token sequence to inspect what one query can access.
Master prompt engineering for large language models: from basic composition to Chain-of-Thought, few-shot, and advanced techniques.
Follow the same Q, K and V through dense and tiled attention. See online softmax preserve the output while avoiding global score and probability matrices.
Explore how prompt token positions connect through causal self-attention, and why connectivity and attention weights do not establish causal influence.
Trace a cached decode step, separate saved projections from growing attention work, and calculate KV-cache memory with explicit dimensions.
Explore neural scaling laws in deep learning: power law relationships between model size, data, and compute that predict AI performance.
Trace ranked BPE merges from text to pieces and integer IDs. Compare BPE, WordPiece and Unigram, and separate normalization from the SentencePiece framework.
A small draft model proposes tokens, the target model checks them all in one pass, and a rejection rule keeps the output distribution exactly the target's.
CLIP, BLIP-2, LLaVA and Flamingo align images with text by different routes. Compare their objectives, bridges, data, and which parts train or stay frozen.