Context Windows: The Memory Limits of LLMs
Interactive visualization of LLM context windows - sliding windows, expanding contexts, and attention patterns that define model memory limits.
No direct links0 refs
🤖
Large Language Models
Architecture, training, and optimization techniques for large language models and transformers.
5
Concepts
All Large Language Models Concepts
Flash Attention: IO-Aware Exact Attention
Interactive Flash Attention visualization - the IO-aware algorithm achieving memory-efficient exact attention through tiling and kernel fusion.
No direct links0 refs
KV Cache: The Secret to Fast LLM Inference
Interactive KV cache visualization - how key-value caching in LLM transformers enables fast text generation without quadratic recomputation.
No direct links0 refs
Tokenization: Converting Text to Numbers
Interactive exploration of tokenization methods in LLMs - BPE, SentencePiece, and WordPiece. Understand how text becomes tokens that models can process.
No direct links0 refs
Mixture of Experts (MoE)
Understanding sparse mixture of experts models - architecture, routing mechanisms, load balancing, and efficient scaling strategies for large language models
No direct links0 refs