Emergent Abilities in Large Language Models
Explore emergent abilities in large language models: sudden capabilities at scale thresholds, phase transitions, and the mirage debate.
5 min readConcept
Explore machine learning concepts related to Scaling. Clear explanations and practical insights.
Explore emergent abilities in large language models: sudden capabilities at scale thresholds, phase transitions, and the mirage debate.
Explore neural scaling laws in deep learning: power law relationships between model size, data, and compute that predict AI performance.
Trace top-k expert routing, normalized output weights and capacity overflow, then compare stored parameters and expert work with explicit compute baselines.