Tagged with

gpu interconnect

Explore machine learning concepts related to gpu interconnect. Clear explanations and practical insights.

Concepts Found

Concepts Related to gpu interconnect

January 29, 2025

Multi-GPU Communication: NVLink, PCIe, and NCCL

How GPUs talk: the bandwidth cliff from HBM to Ethernet, NVLink 5 and GB200 NVL72 topologies, ring AllReduce step by step, and choosing between NCCL, Gloo, and MPI.

multi-GPU NVLink PCIe NCCL distributed training GPU interconnect NVSwitch AllReduce InfiniBand

11 min readConcept