Multi-GPU Communication: NVLink, PCIe, and NCCL
How GPUs talk: the bandwidth cliff from HBM to Ethernet, NVLink 5 and GB200 NVL72 topologies, ring AllReduce step by step, and choosing between NCCL, Gloo, and MPI.
14 min readConcept
Tagged with
InfiniBand
Explore machine learning concepts related to InfiniBand. Clear explanations and practical insights.
1
Concepts Found