Transformers & LLMs

Attention mechanisms, large language models, and multimodal architectures: the building blocks of modern AI.

Attention basics

Start with tokens, follow an attention calculation, then decide where queries can read.

Starting knowledge: Vectors, matrix multiplication and probability distributions

Select a concept. Explore its connections, or open the full explanation.

Explore how text becomes token IDs before following the computations over their embeddings.

Build the scaled dot-product calculation, run it across heads, then block future keys for causal prediction.

Use the same attention calculation with queries from one sequence and keys and values from another.

Turn text into model inputs

Trace BPE merges and compare tokenization methods, normalization and the token budget.

Lines show suggested reading within a branch. Connections offer optional ways to explore; they are not a checklist of prerequisites.