GPU Memory Hierarchy & Optimization
Master GPU memory hierarchy from registers to global memory, understand coalescing patterns, bank conflicts, and optimization strategies for maximum performance
8 min readConcept
Tagged with
cuda
Explore machine learning concepts related to cuda. Clear explanations and practical insights.
8
Concepts Found
Concepts Related to cuda
NVIDIA Unified Virtual Memory
NVIDIA Unified Virtual Memory (UVM): on-demand page migration, memory oversubscription, and simplified CPU-GPU memory management.
12 min readConcept
Pinned Memory and DMA Transfers in PyTorch
Complete guide to PyTorch pin_memory — how DMA transfers work, when pinning helps vs hurts, NUMA effects, profiling with torch.profiler, num_workers interaction, and debugging slow data loading.
8 min readConcept
CUDA Multi-Process Service (MPS): GPU Sharing for Concurrent Workloads
Complete guide to CUDA MPS — architecture, performance benchmarks vs time-slicing and MIG, thread percentage planning, production deployment with systemd and Kubernetes, profiling with nsys, and troubleshooting.
10 min readConcept
NVIDIA Device Files in /dev/
Understanding character devices, major/minor numbers, and the device file hierarchy created by NVIDIA drivers for GPU access in Linux.
11 min readConcept
How Docker Works with GPUs: Device Files, Bind Mounts, and Driver Stacks
Understand how containerized processes access GPU hardware through device files, bind mounts, and the NVIDIA container runtime. Learn the kernel driver vs user-space library distinction.
7 min readConcept
Tensor Cores: Accelerating Deep Learning
NVIDIA Tensor Cores explained: mixed-precision matrix operations delivering 10x speedups for AI training and inference on CUDA GPUs.
6 min readConcept
GPU Streaming Multiprocessor (SM)
Deep dive into the fundamental processing unit of modern GPUs - the Streaming Multiprocessor architecture, execution model, and memory hierarchy
11 min readConcept