High Bandwidth Memory (HBM)
High Bandwidth Memory (HBM) architecture: 3D-stacked DRAM with TSV technology powering NVIDIA GPUs and AI accelerators with TB/s bandwidth.
11 min readConcept
Tagged with
gpu
Explore machine learning concepts related to gpu. Clear explanations and practical insights.
18
Concepts Found
Concepts Related to gpu
GPU Memory Hierarchy & Optimization
Master GPU memory hierarchy from registers to global memory, understand coalescing patterns, bank conflicts, and optimization strategies for maximum performance
8 min readConcept
NVIDIA Unified Virtual Memory
NVIDIA Unified Virtual Memory (UVM): on-demand page migration, memory oversubscription, and simplified CPU-GPU memory management.
12 min readConcept
Page Migration & Fault Handling
CUDA page migration and fault handling between CPU and GPU memory. Learn TLB management, DMA transfers, and memory optimization.
9 min readConcept
NVIDIA Device Files in /dev/
Understanding character devices, major/minor numbers, and the device file hierarchy created by NVIDIA drivers for GPU access in Linux.
11 min readConcept
Distributed Parallelism in Deep Learning
GPU distributed parallelism: Data Parallel (DDP), Tensor Parallel, Pipeline Parallel, and ZeRO optimization for training large AI models.
10 min readConcept
Understanding nvidia-modeset: Kernel Mode-Setting for NVIDIA GPUs
Learn nvidia-modeset for display configuration on Linux. Understand kernel mode-setting, DRM integration, and GPU drivers.
8 min readConcept
CUDA Multi-Process Service (MPS)
Learn CUDA Multi-Process Service (MPS) for GPU sharing. Enable concurrent kernel execution from multiple processes and maximize GPU utilization.
7 min readConcept
Understanding NVIDIA Kubernetes GPU Operator
Automate NVIDIA GPU management in Kubernetes with the GPU Operator. Deploy drivers, device plugins, and monitoring as DaemonSets.
8 min readConcept
Understanding CUDA Contexts
Explore the concept of CUDA contexts, their role in managing GPU resources, and how they enable parallel execution across multiple CPU threads.
3 min readConcept
SoA vs AoS: Data Layout Optimization
Master Structure of Arrays (SoA) vs Array of Structures (AoS) data layouts for optimal cache efficiency, SIMD vectorization, and GPU memory coalescing.
6 min readConcept
Understanding NVIDIA Persistence Daemon
Eliminating GPU initialization latency through nvidia-persistenced - a userspace daemon that maintains GPU driver state for optimal startup performance.
11 min readConcept
Flash Attention: IO-Aware Exact Attention
Interactive Flash Attention visualization - the IO-aware algorithm achieving memory-efficient exact attention through tiling and kernel fusion.
7 min readConcept
NCCL: High-Performance Multi-GPU Communication
Master NVIDIA NCCL for multi-GPU deep learning. Learn AllReduce, ring algorithms, and GPU-Direct communication for efficient distributed training on CUDA.
8 min readConcept
PyTorch DataLoader Pipeline
Understanding how PyTorch DataLoader moves data from disk through CPU to GPU, including Dataset, Sampler, Workers, and Collate components.
4 min readConcept
Pinned Memory and DMA Transfers
Understanding PyTorch pin_memory for faster CPU to GPU data transfers using DMA (Direct Memory Access) and page-locked memory.
5 min readConcept
Tensor Cores: Accelerating Deep Learning
NVIDIA Tensor Cores explained: mixed-precision matrix operations delivering 10x speedups for AI training and inference on CUDA GPUs.
6 min readConcept
GPU Streaming Multiprocessor (SM)
Deep dive into the fundamental processing unit of modern GPUs - the Streaming Multiprocessor architecture, execution model, and memory hierarchy
8 min readConcept