Multi-GPU Communication: NVLink, PCIe, and NCCL
How GPUs talk: the bandwidth cliff from HBM to Ethernet, NVLink 5 and GB200 NVL72 topologies, ring AllReduce step by step, and choosing between NCCL, Gloo, and MPI.
14 min readConcept
Tagged with
NVSwitch
Explore machine learning concepts related to NVSwitch. Clear explanations and practical insights.
1
Concepts Found