# Abhik's Technical Portfolio & Blog > A comprehensive portfolio and technical blog specializing in Computer Vision, Deep Learning, and AI systems optimization. Features 100+ interactive visualizations, in-depth research paper analysis, and practical ML engineering guides. > Full version: https://www.abhik.ai/llms-full.txt ## Overview This website serves as a professional portfolio and technical blog platform built with Next.js 15.5.24 (Pages Router), React, and TailwindCSS. It showcases expertise through in-depth articles, research paper reviews, interactive concept explanations, and practical ML engineering insights. ## Key Information - **Author**: Abhik Sarkar - **Role**: Machine Learning Engineer - **Website**: https://www.abhik.ai - **GitHub**: @abhiksark - **Twitter**: @abhiksark - **LinkedIn**: /in/abhiksark - **License**: Content may be cited with attribution - **Version**: 0.38.0 - **Last Updated**: 2026-11-10 - **Primary Focus**: Machine Learning Engineering & Computer Vision - **Tech Stack**: Next.js 15.5.24 (Pages Router), React 18.2.0, TailwindCSS 3.4.19, MDX - **Content Types**: Technical Articles (30), Research Papers (89), Interactive Concepts (222), Talks (7) ## Architecture & Technical Details - **Framework**: Next.js 15.5.24 with **Pages Router** (NOT App Router) - **Language**: JavaScript/JSX (.js, .jsx files) - **Styling**: TailwindCSS 3.4.19 exclusively (no CSS-in-JS) - **Content System**: MDX-based with interactive React components - **Deployment**: Vercel (a Docker multi-stage build with output: 'standalone' is kept for self-hosting) - **Analytics**: Vercel Analytics & Speed Insights - **SEO**: Unified SEO system with dynamic OG images rendered at /og - **Interactive Features**: 100+ custom visualizations with progressive loading - **Component Patterns**: Compound components, dark mode via classes, responsive canvases ## Directory Structure - `/src/pages/*` - Next.js pages and routing (Pages Router) - `/src/components/*` - React components (layouts, articles, concepts, icons) - `/src/lib/*` - Utility functions (SEO, metadata, content fetching, image optimization) - `/src/data/*` - Static data (siteMeta, resume, categories) - `/src/styles/*` - CSS and styling (Tailwind, Prism, typography) - `/public/*` - Static assets (images, fonts, RSS feeds, sitemaps) ## Pages - [Home](https://www.abhik.ai): Landing page and introduction - [About](https://www.abhik.ai/about): Professional background and expertise - [Articles](https://www.abhik.ai/articles): Technical articles and insights - [Papers](https://www.abhik.ai/papers): Research paper analysis and reviews - [Concepts](https://www.abhik.ai/concepts): Interactive ML, GPU and systems concepts - [Speaking](https://www.abhik.ai/speaking): Conference talks and workshops - [Resume](https://www.abhik.ai/resume): Professional resume and experience - [Consulting](https://www.abhik.ai/consulting): ML and computer vision consulting services - [Expertise](https://www.abhik.ai/expertise): Index of canonical references by topic - [Comparisons](https://www.abhik.ai/comparisons): Head-to-head technical comparisons - [Uses](https://www.abhik.ai/uses): Tools, software and hardware in daily use - [Bookmarks](https://www.abhik.ai/bookmarks): Curated resources and links - [Privacy](https://www.abhik.ai/privacy): Privacy policy ## Technical Articles (30) - [One multiply, one RTX 5090, eight kernels](https://www.abhik.ai/articles/rtx-5090-matmul-eight-kernels): One small-batch matrix multiply on a current NVIDIA GPU, followed from 748 microseconds to 29 across eight measured CUDA kernels, with the code, the machine instructions and the profiler counters that changed at each step. - [Vision Transformer Deep Dive: ViT to Modern Descendants](https://www.abhik.ai/articles/vit-deep-dive): Build ViT from pixels, then trace DeiT, Swin, DINOv2, SAM, LLaVA, and practical latency optimizations. - [Xid 31 MMU Faults: What Causes Them and How to Fix Production GPU Crashes](https://www.abhik.ai/articles/gpu-xid31-mmu-faults): Deep dive into NVIDIA GPU Xid 31 MMU faults: how GPU virtual memory works, what causes page table walk failures, and how we eliminated 28 daily crashes in a production video pipeline processing 7,000+ videos. - [The Complete NVIDIA Xid Error Field Guide](https://www.abhik.ai/articles/nvidia-xid-errors): Diagnose NVIDIA Xid errors with incident-response steps, recovery guidance, worked debugging examples, and a searchable reference for common GPU faults. - [CUDA Matrix Multiplication Optimization: From Naive to Near-cuBLAS](https://www.abhik.ai/articles/cuda-matrix-multiplication-optimization): Step-by-step CUDA matrix multiplication optimization with 9 interactive visualizations. From naive kernels through shared memory tiling to near-cuBLAS speeds. - [Numerical Sensitivity: Why FP16 Breaks NAdam](https://www.abhik.ai/articles/numerical-sensitivity): Visual exploration of floating-point arithmetic and numerical stability. Learn why NAdam fails in FP16 and how machine epsilon affects deep learning. - [SAM's Multi-Mask Ambiguity: A Visual Deep Dive](https://www.abhik.ai/articles/sam-multi-mask-ambiguity): Deep dive into how SAM resolves point prompt ambiguity through three-mask output design, IoU prediction, and intelligent mode switching. - [YOLOv11 Loss Functions Explained: Interactive Visual Guide](https://www.abhik.ai/articles/visualizing-yolov11): Understand YOLOv11's loss functions through interactive visualizations. Compare IoU variants (GIoU, DIoU, CIoU), explore Distribution Focal Loss (DFL), and see why anchor-free detection matters. - [H.264 Implementation & Applications (Part 3 of 3)](https://www.abhik.ai/articles/h264-implementation-applications): H.264 Part 3: Implementation guide covering profiles, levels, hardware vs software encoding, and real-world video compression applications. - [H.264 Transform & Quantization (Part 2 of 3)](https://www.abhik.ai/articles/h264-transform-quantization): H.264 Part 2: Dive into DCT transforms, quantization strategies, rate-distortion optimization, and entropy coding at the mathematical heart of compression. - [H.264 Codec Fundamentals: How Video Compression Works (Part 1 of 3)](https://www.abhik.ai/articles/h264-fundamentals): How the H.264 codec compresses video, explained Part 1 of 3: video compression fundamentals, the core encoding pipeline, block-based macroblock processing, and motion estimation — with interactive demos. - [Adopting Zettelkasten for Paper Explanations](https://www.abhik.ai/articles/zettel): Announcing the adoption of the Zettelkasten method for structuring paper explanations on abhik.ai/papers to improve connections and reduce redundancy - [How torch.compile Generates Optimized GPU Kernels: Fusion, Tiling, and Shape Specialization](https://www.abhik.ai/articles/compiling-pytorch-kernel): How torch.compile generates optimized GPU kernels from PyTorch eager code: kernel fusion, memory tiling, shape specialization, and the TorchInductor backend — with interactive visualizations of every transformation. - [Fix PyTorch "view size is not compatible" Error](https://www.abhik.ai/articles/view-size-not-compatible): Learn why PyTorch throws the "view size is not compatible" error, understand tensor memory layout, and discover optimal solutions with performance benchmarks. - [GPU Boot Errors: initramfs and Driver Conflicts](https://www.abhik.ai/articles/gpu-boot-errors): Fix Linux GPU boot errors: nouveau vs NVIDIA driver conflicts, initramfs solutions, and the early driver loading chicken-and-egg problem. - [Interactive H.264 Guide: Video Compression Visuals](https://www.abhik.ai/articles/h264-interactive-guide): Interactive H.264 video compression guide with visualizations. Explore motion estimation, DCT transforms, quantization, and rate-distortion optimization. - [GGML File Structure: Tensor Layout and Quantized Model Format Explained](https://www.abhik.ai/articles/ggml-structure): Understand GGML file structure and quantization formats used by local LLMs. Visual guide to how llama.cpp stores and loads model weights efficiently. - [Quantization Deep Dive: From FP32 to INT4](https://www.abhik.ai/articles/quantization-deep-dive): Master neural network quantization with interactive visualizations. Explore QAT, PTQ, GPTQ, AWQ, and SmoothQuant methods for efficient model deployment. - [How TensorRT Works: NVIDIA Inference Optimization](https://www.abhik.ai/articles/how-tensorrt-works): Explore TensorRT optimization: layer fusion, INT8 quantization, kernel auto-tuning, and deployment strategies with 8+ interactive visualizations. - [Kernel Fusion in Deep Learning: How GPU Kernels Are Merged](https://www.abhik.ai/articles/kernel-fusion): Kernel fusion merges multiple neural network operations into a single GPU kernel to eliminate intermediate memory writes — this article explains how fusion works, why it helps deep learning workloads, and how TensorRT and torch.compile use it. - [Python's logging Module Is Already a Singleton](https://www.abhik.ai/articles/python-production-logging): You don't need a logging framework, registry, or factory. Python's stdlib logging is a global singleton — getLogger(name) always returns the same instance, and dictConfig can configure any logger in the process, including third-party libraries. - [H.264 vs H.265 vs AV1: Comparing Modern Video Codecs](https://www.abhik.ai/articles/h264-vs-h265-vs-av1): Compare H.264, H.265, and AV1 for bitrate, encoding cost, hardware encode and decode, and licensing. Learn how to test a codec on your delivery path. - [CPython Internals: How Python Really Works Under the Hood](https://www.abhik.ai/articles/cpython-internals): Deep dive into CPython internals: bytecode compilation, memory management, the GIL, object model, and garbage collection. - [C++ Compilation Process: From Source Code to Object Files](https://www.abhik.ai/articles/cpp-compilation-process): How C++ compilers transform source code through preprocessing, parsing, optimization, and code generation. Interactive visualizations included. - [C++ Linking: Static, Dynamic, and Everything Between](https://www.abhik.ai/articles/cpp-linking-in-depth): Master the linking process in C++ including symbol resolution, static vs dynamic linking, relocations, GOT/PLT, and solving common linking errors. - [C++ Loading and Runtime: From Executable to Process](https://www.abhik.ai/articles/cpp-loading-runtime): Explore how C++ programs load into memory: dynamic linking at runtime, memory layout, and the complete startup sequence. - [Dynamic Model Loading: The Registry Pattern](https://www.abhik.ai/articles/registry-pattern): Master dynamic model loading in PyTorch using the Registry Pattern. Learn decorators and configuration-driven architecture from OpenMMLab. - [The Magic Behind FF D8 FF: How Magic Numbers Define File Formats](https://www.abhik.ai/articles/magic-numbers): Explore the fascinating concept of magic numbers in computing and discover how FF D8 FF serves as the file signature for JPEG files. - [Understanding Image Encoding: Lossy vs. Lossless Compression](https://www.abhik.ai/articles/image-encoding): Explore image encoding fundamentals: lossy vs lossless compression, JPEG and PNG techniques, DCT transforms, and how digital images are compressed. - [ASCII vs UTF-8 vs UTF-16 vs UTF-32: A Comparison](https://www.abhik.ai/articles/text-encoding): Compare ASCII, UTF-8, UTF-16, and UTF-32 encodings. Learn why character encoding matters for LLMs, compatibility, and text processing. ## Research Papers (89) - [MPK: A Compiler and Runtime for Mega-Kernelizing Tensor Programs](https://www.abhik.ai/papers/mpk): Follow how MPK turns a model into one persistent GPU kernel by tracking dependencies between tasks on individual SMs, with a task-graph explorer, a cross-task pipelining timeline, and a runnable CPU model of its compiler stages. - [Welder: Scheduling Deep Learning Memory Access via Tile-graph](https://www.abhik.ai/papers/welder): Follow how Welder propagates an output tile back through a chain of operators, counts tile traffic analytically, and chooses where each intermediate tile lives, with each reported speedup tied to its stated condition. - [Liger Kernel: Fused RMSNorm](https://www.abhik.ai/papers/liger-rmsnorm): Derive RMSNorm, trace fused rowwise normalization, inspect mixed-precision cast boundaries, and follow the input and shared-weight gradients in Liger Kernel. - [Liger Kernel: Fused SwiGLU](https://www.abhik.ai/papers/liger-swiglu): Trace Liger Kernel’s fused SiLU and multiply, derive both SwiGLU gradients, and separate saved activation memory from measured GPU performance. - [Online Normalizer Calculation for Softmax](https://www.abhik.ai/papers/online-softmax): Derive the online softmax recurrence, merge parallel summaries, count memory passes, and test a runnable reference against stable softmax. - [Adam: A Method for Stochastic Optimization](https://www.abhik.ai/papers/adam): Follow Adam’s first and second moment estimates, derive zero-initialization bias correction, inspect epsilon placement, and separate useful optimization practice from convergence guarantees. - [Learning Representations by Back-Propagating Errors](https://www.abhik.ai/papers/backpropagation): Follow the forward trace, sum reverse gradient paths, and check every weight and bias with finite differences while reading the landmark 1986 representation-learning experiments. - [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://www.abhik.ai/papers/batch-normalization): Trace mini-batch statistics, gradients shared across samples, learned scale and shift, and frozen inference; then inspect the original Inception evidence and the limits of its causal explanation. - [Dropout: A Simple Way to Prevent Neural Networks from Overfitting](https://www.abhik.ai/papers/dropout): Explore Bernoulli subnetworks, original versus inverted scaling, exact small-mask expectations, and the gap between linear expectation matching and averaging nonlinear networks. - [Focal Loss for Dense Object Detection](https://www.abhik.ai/papers/focal-loss): Separate alpha weighting from focal modulation, derive the complete logit gradient, and inspect population imbalance, prior initialization and RetinaNet evidence. - [Feature Pyramid Networks: Semantics at Every Scale](https://www.abhik.ai/papers/fpn): Build FPN from C2–C5, lateral 1×1 projections, nearest top-down addition and 3×3 outputs; trace RoI level assignment and the original detection evidence. - [LeNet: Gradient-Based Learning Applied to Document Recognition](https://www.abhik.ai/papers/lenet): Read the original LeNet-5 through shared filters, learned subsampling, partial C3 connectivity and fixed RBF outputs, with executable references and the 1998 document-recognition evidence. - [Long Short-Term Memory: The Original 1997 LSTM](https://www.abhik.ai/papers/lstm): Explore the original no-forget-gate LSTM: gated writes and reads, the constant error carousel, path derivatives and the delayed-sequence experiments behind the 1997 paper. - [Mask R-CNN: Aligned Features for Instance Segmentation](https://www.abhik.ai/papers/mask-rcnn): Trace continuous RoIAlign sample coordinates, bilinear interpolation and matched-class mask loss, then examine the original Mask R-CNN evidence and limits. - [SIFT: Distinctive Image Features from Scale-Invariant Keypoints](https://www.abhik.ai/papers/sift): Trace SIFT from extrema in scale space to orientation-normalized 128-dimensional descriptors and ambiguity-aware matching, with original PDF excerpts and inspectable reference operations. - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://www.abhik.ai/papers/u-net): Reconstruct the original 572-to-388 valid-convolution U-Net, crop-and-concatenate skips, overlap tiling and boundary-weighted training, with executable geometry. - [VGG: Very Deep Convolutional Networks for Large-Scale Image Recognition](https://www.abhik.ai/papers/vgg): Explore VGG’s 3×3 stacks, exact VGG-16/VGG-19 shapes and parameter counts, then separate architecture effects from scale, crop and ensemble evaluation in the original study. - [D-FINE: Refine the Distribution Behind the Box](https://www.abhik.ai/papers/d-fine): Explore D-FINE’s edge distributions, iterative refinement, localization loss and self-distillation with interactive figures, Python and benchmark caveats. - [EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba](https://www.abhik.ai/papers/efficientvmamba): Trace spatial residue routing, sparse state updates and inverse merging, then examine local fusion, inverted stage allocation and the limits of EfficientVMamba’s efficiency evidence. - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://www.abhik.ai/papers/grounding-dino): Trace Grounding DINO through text-image enhancement, language-guided anchors, cross-modal decoding and phrase grounding, with Python and evaluation caveats. - [Mamba-3: Improved Sequence Modeling using State Space Principles](https://www.abhik.ai/papers/mamba3): Derive Mamba-3’s two-endpoint recurrence, data-dependent rotations and MIMO state writes, then examine its state-tracking, retrieval and inference evidence. - [MambaVision: A Hybrid Mamba-Transformer Vision Backbone](https://www.abhik.ai/papers/mambavision): Trace MambaVision through its hierarchy, symmetric mixer paths, selective recurrence and windowed attention, with executable Python and a critical reading of the evidence. - [RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer](https://www.abhik.ai/papers/rt-detrv2): RT-DETRv2 explained through per-scale sampling, discrete operators, training schedules, executable Python mechanics and fair latency comparisons. - [S4: Efficiently Modeling Long Sequences with Structured State Spaces](https://www.abhik.ai/papers/s4): From HiPPO memory to bilinear dynamics, equivalent recurrent and convolutional execution, and the DPLR kernel construction that makes S4 practical. - [S5: Simplified State Space Layers for Sequence Modeling](https://www.abhik.ai/papers/s5): Derive S5 from S4: shared MIMO state, diagonal dynamics, exact discretization, associative scans and HiPPO initialization, with interactive diagrams and executable Python. - [VideoMamba: State Space Model for Efficient Video Understanding](https://www.abhik.ai/papers/videomamba): Explore VideoMamba’s video tokens, bidirectional scans, self-distillation and masked pretraining through interactive diagrams and protocol-aware benchmarks. - [Deformable DETR: Learn Where to Read](https://www.abhik.ai/papers/deformable-detr): A visual explanation of sparse multi-scale attention, bilinear sampling, point and box references, and optional refinement in Deformable DETR. - [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://www.abhik.ai/papers/dino-detr): Explore DINO detector queries, denoising masks and look-forward-twice gradients through interactive diagrams, source equations and runnable Python. - [LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection](https://www.abhik.ai/papers/lw-detr): How LW-DETR combines window-major ViT features, depth aggregation, a shallow decoder and stronger training, with a careful reading of latency evidence. - [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://www.abhik.ai/papers/mamba): Trace Mamba through input-dependent state updates, associative scanning, the complete block, persistent decode buffers and runnable CPU code. - [Mamba-2: Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://www.abhik.ai/papers/mamba2): Derive Mamba-2 from scalar state updates to a causal mixing matrix, chunked SSD, complete block structure and runnable CPU references. - [RF-DETR: Neural Architecture Search for Real-Time Detection Transformers](https://www.abhik.ai/papers/rf-detr): Explore RF-DETR through recorded detections, architecture search, query budgets and masks, with runnable CPU calculations and source-backed paper results. - [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://www.abhik.ai/papers/siglip2): Explore SigLIP 2 through decoder supervision, EMA teachers, masked features and NaFlex geometry, with runnable CPU mechanics and recorded image-text matching. - [Vision Mamba: Bidirectional State Space Models for Images](https://www.abhik.ai/papers/vision-mamba): Follow Vision Mamba from image patches through selective state updates, two aligned scan directions, a constant-resolution backbone, and runnable Python. - [VMamba: Visual State Space Model](https://www.abhik.ai/papers/vmamba): Follow VMamba from a compact running state through four image scan orders, aligned cross-merge, the final VSS block, and runnable Python. - [HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees](https://www.abhik.ai/papers/hived): HiveD reserves groups of GPUs that sit together, not GPU counts, so a tenant never queues longer in a shared cluster than in a private one of the same size. - [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://www.abhik.ai/papers/megatron-lm): Megatron-LM splits each transformer layer across GPUs with two all-reduces forward and two backward, and trains an 8.3B-parameter GPT-2 on 512 V100s. - [Collective Communication for 100k+ GPUs](https://www.abhik.ai/papers/ncclx): NCCLX is Meta's communication stack for Llama 4: host-driven, zero-copy collectives that start a 96K-GPU job 11x faster and cut decode time by up to 83%. - [Hilbert-Guided Sparse Local Attention](https://www.abhik.ai/papers/hilbert-local-attention): Reordering image tokens along a Hilbert curve makes 2D local attention block-sparse, for about 4x faster window attention and 18x faster slide attention. - [VIOLIN: Spatial Priors via Space Filling Curves for ViTs](https://www.abhik.ai/papers/violin): VIOLIN multiplies ViT attention by decay masks from eight space-filling curves: 1,296 extra parameters on DeiT-B and up to 8.7 points on VTAB-1K spatial tasks. - [Every µs Matters: Achieving Near Speed-of-Light Latency in GPU Collectives](https://www.abhik.ai/papers/every-microsecond-matters): How the Every µs Matters paper brings small GPU AllReduce operations close to the hardware speed-of-light bound: it measures a 1.404 µs floor on GB200, removes memory barriers with LL, sentinel and LL128 atomic signalling, and cuts vLLM inter-token latency by up to 13%. - [NanoFlow: Towards Optimal Large Language Model Serving Throughput](https://www.abhik.ai/papers/nanoflow): How NanoFlow raises LLM serving throughput by overlapping compute, memory and network work inside a single GPU: it shows serving is compute-bound, splits each batch into nano-batches, and shares SMs between concurrent kernels, reaching 1.91x the throughput of TensorRT-LLM. - [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://www.abhik.ai/papers/flashattention): How FlashAttention computes exact attention with linear memory by tiling Q, K, and V into SRAM-resident blocks and fusing the softmax, avoiding the quadratic HBM cost of materializing the full attention matrix. - [LoRA: Low-Rank Adaptation of Large Language Models](https://www.abhik.ai/papers/lora): How LoRA adapts a frozen large language model by learning a low-rank update ΔW = (α/r)·BA to each weight matrix, training under 1% of the parameters of full fine-tuning and adding zero inference latency once merged. - [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://www.abhik.ai/papers/paged-attention): How PagedAttention (the memory manager behind vLLM) applies OS-style virtual-memory paging to the KV cache — fixed-size blocks, a block table, and copy-on-write prefix sharing — to reduce final-block waste, remove external block fragmentation, and improve throughput in the paper’s measured serving workloads. - [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://www.abhik.ai/papers/switch-transformer): How the Switch Transformer scales to trillions of parameters by routing each token to a single expert (top-1 gating), decoupling model capacity from per-token compute while managing expert load with a capacity factor. - [AlexNet: ImageNet Classification with Deep Convolutional Neural Networks](https://www.abhik.ai/papers/alexnet): How AlexNet won ImageNet 2012 by a landslide and launched the deep-learning era: a deep convolutional network trained on two GPUs with ReLU activations, dropout regularization, and aggressive data augmentation. - [ALIGN: Scaling Vision-Language Learning with Noisy Text Supervision](https://www.abhik.ai/papers/align): How ALIGN scales vision-language pre-training to 1.8 billion noisy image-alt-text pairs, showing that the sheer scale of a raw, uncurated dataset can outweigh careful data cleaning for a simple dual-encoder contrastive model. - [BERT: Pre-training of Deep Bidirectional Transformers](https://www.abhik.ai/papers/bert): How BERT pre-trains a deep bidirectional Transformer with masked language modeling and next-sentence prediction, then fine-tunes the same model to state-of-the-art results across a wide range of NLP tasks. - [BLIP: Bootstrapping Language-Image Pre-training](https://www.abhik.ai/papers/blip): How BLIP unifies vision-language understanding and generation in one model, and bootstraps noisy web data with CapFilt — a captioner that synthesizes captions and a filter that removes noisy image-text pairs. - [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://www.abhik.ai/papers/coca): How CoCa trains a single image-text foundation model with both a contrastive loss and a generative captioning loss by splitting the text decoder into unimodal and multimodal halves, computed in one forward pass. - [Flamingo: a Visual Language Model for Few-Shot Learning](https://www.abhik.ai/papers/flamingo): How Flamingo bridges a frozen vision encoder and a frozen language model with a Perceiver Resampler and gated cross-attention layers, enabling few-shot learning from interleaved image-text sequences. - [Florence-2: A Unified Model for Many Vision Tasks](https://www.abhik.ai/papers/florence-2): How Florence-2 turns detection, segmentation, grounding, captioning, and OCR into one sequence-to-sequence problem driven by task prompts, expressing spatial outputs as location tokens in the text vocabulary. - [GAN: Generative Adversarial Networks](https://www.abhik.ai/papers/gan): How GANs frame generative modeling as a two-player minimax game: a generator turns noise into samples while a discriminator learns to tell real from fake, driving the generator toward the true data distribution. - [Qwen2-VL: Vision-Language Perception at Any Resolution](https://www.abhik.ai/papers/qwen2-vl): How Qwen2-VL perceives images and video at any resolution with naive dynamic resolution (variable visual tokens) and M-RoPE, a multimodal rotary position embedding that decomposes position into temporal, height, and width components. - [SAM 2: Segment Anything in Images and Videos](https://www.abhik.ai/papers/sam-2): How SAM 2 extends promptable segmentation from images to video with a streaming memory bank and memory attention, propagating masks across frames and recovering objects after occlusion. - [SigLIP: Sigmoid Loss for Language Image Pre-Training](https://www.abhik.ai/papers/siglip): How SigLIP replaces CLIP’s softmax contrastive loss with a pairwise sigmoid loss, removing the global softmax denominator and enabling chunked all-pairs computation and strong language-image pre-training at smaller batch sizes. - [VAE: Auto-Encoding Variational Bayes](https://www.abhik.ai/papers/vae): How the variational autoencoder turns an autoencoder into a generative model: an encoder predicts a Gaussian latent distribution, the reparameterization trick makes sampling differentiable, and the ELBO balances reconstruction against a KL prior. - [DDPM: Denoising Diffusion Probabilistic Models](https://www.abhik.ai/papers/ddpm): How diffusion models learn to generate images by reversing a gradual noising process — the foundation of Stable Diffusion, DALL-E, and modern image generation. - [Flow Matching: Simplified Generative Modeling](https://www.abhik.ai/papers/flow-matching): How Flow Matching simplifies generative modeling by learning straight transport paths from noise to data — faster sampling, simpler training, and the foundation of modern generation systems. - [Latent Diffusion Models: High-Resolution Image Synthesis](https://www.abhik.ai/papers/latent-diffusion): How Latent Diffusion Models made high-resolution image generation practical by moving diffusion to a compressed latent space — the architecture behind Stable Diffusion. - [BEiT: BERT Pre-Training of Image Transformers](https://www.abhik.ai/papers/beit): BEiT masked image modeling explained through original paper sources, exact PyTorch code, frozen DALL-E targets, and real masked-token and classification outputs. - [DINOv2: Learning Robust Visual Features without Supervision](https://www.abhik.ai/papers/dinov2): How DINOv2 combines DINO self-distillation with iBOT masked prediction on curated LVD-142M data, with source-defined frozen-feature evaluations for classification, segmentation, depth, and retrieval. - [I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://www.abhik.ai/papers/ijepa): Follow I-JEPA from the original paper and equations to runnable Python, then inspect real pretrained predictions of masked image features. - [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://www.abhik.ai/papers/vjepa2): How V-JEPA 2 scales self-supervised video learning to 1M+ hours with masked latent prediction and 3D-RoPE, then extends to V-JEPA 2-AC for robotic planning using 62 hours of robot video with recorded actions. - [BYOL: Bootstrap Your Own Latent](https://www.abhik.ai/papers/byol): How self-supervised learning works without negative pairs — a predictor and momentum target network enable learning without a contrastive negative-pair loss. - [DINO: Emerging Properties in Self-Supervised Vision Transformers](https://www.abhik.ai/papers/dino): How self-distillation with no labels produces Vision Transformer attention maps that automatically segment objects — without any pixel-level supervision. - [MAE: Masked Autoencoders Are Scalable Vision Learners](https://www.abhik.ai/papers/mae): How masking 75% of image patches and reconstructing pixels creates a scalable self-supervised learner that trains ViT-H to 87.8% on ImageNet-1K with label-free pre-training and an efficient visible-only encoder. - [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning](https://www.abhik.ai/papers/moco): How a momentum-updated encoder and a dictionary queue make contrastive learning practical — large dictionaries with consistent keys, no large-batch requirement. - [SimCLR: A Simple Framework for Contrastive Learning](https://www.abhik.ai/papers/simclr): How a simple framework — augmentation, shared encoder, projection head, and contrastive loss — set a new standard for self-supervised visual representation learning. - [V-JEPA: Learning Video Representations by Predicting in Latent Space](https://www.abhik.ai/papers/vjepa): How V-JEPA learns powerful video representations by predicting masked spatiotemporal regions in embedding space rather than reconstructing pixels, achieving state-of-the-art frozen features with superior label efficiency. - [VICReg: Self-Supervised Learning Without Collapse](https://www.abhik.ai/papers/vicreg): How variance, invariance, and covariance regularization enables self-supervised representation learning without negative pairs or momentum encoders. - [Visual Instruction Tuning](https://www.abhik.ai/papers/visual-instruction-tuning): LLaVA paper: align LLMs with visual information through instruction tuning on image-text pairs, enabling multimodal understanding and reasoning. - [Plain ViT Backbones for Object Detection](https://www.abhik.ai/papers/vit-object-detection): Investigating the effectiveness of plain Vision Transformers as backbones for object detection and proposing modifications to improve their performance. - [You Only Look Once: Unified, Real-Time Object Detection](https://www.abhik.ai/papers/yolo): Introducing YOLO, a unified, real-time object detection system that frames object detection as a single regression problem. - [EfficientNet: Compound Scaling for CNNs](https://www.abhik.ai/papers/efficientnet): Connect EfficientNet’s compound scaling equations to its MBConv architecture, annotated PyTorch implementation, and real pretrained B0 versus B4 predictions. - [Faster R-CNN: Real-Time Object Detection](https://www.abhik.ai/papers/faster-rcnn): Faster R-CNN explained: how Region Proposal Networks (RPN) enable near real-time object detection with shared convolutional features. - [Segment Anything Model (SAM)](https://www.abhik.ai/papers/sam): SAM is a promptable segmentation model that can segment any object in an image using points, boxes, or text prompts with zero-shot generalization. - [End-to-End Object Detection with Transformers](https://www.abhik.ai/papers/DETR): Introducing DETR, a novel end-to-end object detection framework that leverages Transformers to directly predict a set of object bounding boxes. - [BLIP-2: Efficient Vision-Language Pre-training](https://www.abhik.ai/papers/blip2): BLIP-2 leverages frozen image encoders and LLMs for efficient vision-language pre-training, achieving state-of-the-art multimodal performance. - [ViT: An Image is Worth 16x16 Words](https://www.abhik.ai/papers/image-worth-16x16): Vision Transformer explained with original paper equations, tensor shapes, runnable PyTorch code, and real pretrained image classification. - [A Survey of Techniques for Optimizing Transformer Inference](https://www.abhik.ai/papers/optimizing-transformer-inference): Survey of transformer inference optimization: pruning, quantization, knowledge distillation, neural architecture search, and hardware acceleration. - [SURF: Speeded Up Robust Features](https://www.abhik.ai/papers/surf): SURF is a fast and robust algorithm for local feature detection and description, used in object recognition, image registration, and 3D reconstruction. - [Swin Transformer: Hierarchical ViT with Shifted Windows](https://www.abhik.ai/papers/swin-transformer): Swin Transformer: hierarchical Vision Transformer using shifted windows for efficient image classification, object detection, and segmentation. - [CLIP: Visual Models via Language Supervision](https://www.abhik.ai/papers/clip): CLIP explained: contrastive learning on 400M image-text pairs enables zero-shot image classification and powerful vision-language understanding. - [Making Deep Learning Go Brrrr From First Principles](https://www.abhik.ai/papers/deeplearning-go-brr): Deep learning performance optimization from first principles. Learn to identify compute-bound, memory-bound, and overhead bottlenecks with fusion techniques. - [Attention Is All You Need](https://www.abhik.ai/papers/attention-is-all-you-need): Deep dive into the Transformer architecture that revolutionized NLP. Understand self-attention, multi-head attention, and positional encoding. - [Data Movement Is All You Need: Optimizing Transformers](https://www.abhik.ai/papers/data-movement-transformer): Analysis of transformer performance bottlenecks caused by data movement. Learn optimization strategies for memory-bound operations on GPUs. - [Deep Residual Learning for Image Recognition](https://www.abhik.ai/papers/deep-residual-learning): ResNet analysis: how skip connections and residual learning solved the degradation problem, enabling training of 100+ layer neural networks. ## Interactive Concepts (222) ### [Transformers & LLMs](https://www.abhik.ai/concepts/category/transformers) (28) Attention mechanisms, large language models, and multimodal architectures: the building blocks of modern AI. - [CLS Token in Vision Transformers](https://www.abhik.ai/concepts/transformers/cls-token): Trace how a learned CLS row joins image patches, gathers evidence through self-attention, and becomes the image-level classification readout. - [Hierarchical Attention in Vision Transformers](https://www.abhik.ai/concepts/transformers/hierarchical-attention): Trace how local windows, shifted cross-window exchange, and patch merging turn one high-resolution token grid into a multi-scale vision hierarchy. - [Multi-Head Attention](https://www.abhik.ai/concepts/transformers/multihead-attention): How multi-head attention runs scaled dot-product attention in parallel across several representation subspaces to build context-aware token embeddings. - [Positional Embeddings in Vision Transformers](https://www.abhik.ai/concepts/transformers/positional-embeddings-vit): Move the same image patches with and without positional addition to see how learned absolute positions change a Vision Transformer’s attention inputs. - [Self-Attention in Vision Transformers](https://www.abhik.ai/concepts/transformers/self-attention-vit): Follow one image patch through Q/K/V projection, scaled scores, row-wise softmax, value mixing, and the residual update inside a Vision Transformer. - [ALiBi: Attention with Linear Biases](https://www.abhik.ai/concepts/transformers/alibi): Follow a query-key distance through ALiBi head slopes, attention logits and normalized weights, then understand the evidence and limits of length extrapolation. - [Flash Attention vs MHA vs GQA vs MQA: Comparing Attention Mechanisms](https://www.abhik.ai/concepts/transformers/attention-comparison): Compare head sharing with explicit projection dimensions and KV-cache bytes. Separate exact memory ratios from measured quality and latency, and see why FlashAttention is an independent choice. - [Attention Sinks: Stable Streaming LLMs](https://www.abhik.ai/concepts/transformers/attention-sinks): See how preserving initial attention sinks stabilizes a rolling KV cache, why it costs recent slots, and why stable streaming does not mean infinite recall. - [Cross-Attention: Bridging Different Modalities](https://www.abhik.ai/concepts/transformers/cross-attention): Trace a target query through source keys, attention weights, and a weighted value mix. Understand cross-attention origins, shapes, masks, and applications. - [Grouped-Query Attention: Head Sharing and KV Cache](https://www.abhik.ai/concepts/transformers/grouped-query-attention): Trace how grouped-query attention keeps independent query heads while sharing fewer key/value heads, projections, and compact KV-cache rows during LLM decoding. - [Linear Attention Approximations](https://www.abhik.ai/concepts/transformers/linear-attention-approximations): Understand normalized kernel attention, low-rank sequence projections and their trade-offs, and compare them with sparse attention and exact tiled softmax. - [Masked and Causal Attention](https://www.abhik.ai/concepts/transformers/masked-attention): Trace causal attention from shifted next-token labels through the lower-triangular mask, pre-softmax blocking, parallel training, and incremental decoding. - [Multi-Query Attention (MQA)](https://www.abhik.ai/concepts/transformers/multi-query-attention): Trace how multi-query attention keeps independent query heads while sharing one key/value head, with exact cache bytes and a shape-correct decoding example. - [Rotary Position Embeddings (RoPE)](https://www.abhik.ai/concepts/transformers/rotary-position-embeddings): Trace RoPE in radians, see how shared shifts preserve relative-position scores, and distinguish its frequency schedule from long-context guarantees. - [Scaled Dot-Product Attention](https://www.abhik.ai/concepts/transformers/scaled-dot-product): Compare scaled and unscaled attention on the same dot products, then derive the square-root scaling factor and its limits. - [Sliding Window Attention](https://www.abhik.ai/concepts/transformers/sliding-window-attention): Explore causal and symmetric local attention with exact window counts, then distinguish direct keys, multi-layer input reach, and inference KV storage. - [Sparse Attention Patterns](https://www.abhik.ai/concepts/transformers/sparse-attention-patterns): Compose sparse attention patterns, follow one query through masking and softmax, and separate fewer allowed pairs from actual sparse-kernel savings. - [The Vision-Language Alignment Problem](https://www.abhik.ai/concepts/transformers/alignment-problem): Trace one image-caption pair through cosine scores, contrastive positives and negatives, a trainable projection, and retrieval evaluation. - [Context Windows: Capacity and Attention Limits](https://www.abhik.ai/concepts/transformers/context-windows): See how context capacity, attention masks, and KV caches differ. Move a planted fact through a token sequence to inspect what one query can access. - [Flash Attention: IO-Aware Exact Attention](https://www.abhik.ai/concepts/transformers/flash-attention): Follow the same Q, K and V through dense and tiled attention. See online softmax preserve the output while avoiding global score and probability matrices. - [KV Cache: Reuse Past Keys and Values](https://www.abhik.ai/concepts/transformers/kv-cache): Trace a cached decode step, separate saved projections from growing attention work, and calculate KV-cache memory with explicit dimensions. - [The Modality Gap in Multimodal AI](https://www.abhik.ai/concepts/transformers/modality-gap): Measure the centroid gap between normalized image and text embeddings, then check why a smaller gap alone does not guarantee correct retrieval. - [Multimodal Scaling Laws](https://www.abhik.ai/concepts/transformers/scaling-laws): Explore a fixed compute budget with Chinchilla’s text-only loss fit, then see why multimodal scaling needs its own data, objectives, and evaluation metrics. - [Tokenization: Converting Text to Numbers](https://www.abhik.ai/concepts/transformers/tokenization): Trace ranked BPE merges from text to pieces and integer IDs. Compare BPE, WordPiece and Unigram, and separate normalization from the SentencePiece framework. - [Vision-Language Adapters: Efficient Fine-tuning](https://www.abhik.ai/concepts/transformers/vision-language-adapters): Trace frozen weights and trainable LoRA matrices, calculate adapter parameter counts, and compare bottleneck adapters, prefix tuning and BitFit for vision-language models. - [Mixture of Experts (MoE)](https://www.abhik.ai/concepts/transformers/mixture-of-experts): Trace top-k expert routing, normalized output weights and capacity overflow, then compare stored parameters and expert work with explicit compute baselines. - [Speculative Decoding: Draft, Then Verify](https://www.abhik.ai/concepts/transformers/speculative-decoding): A small draft model proposes tokens, the target model checks them all in one pass, and a rejection rule keeps the output distribution exactly the target's. - [Vision-Language Alignment: CLIP, BLIP-2, LLaVA, Flamingo](https://www.abhik.ai/concepts/transformers/vision-language-alignment): CLIP, BLIP-2, LLaVA and Flamingo align images with text by different routes. Compare their objectives, bridges, data, and which parts train or stay frozen. ### [Deep Learning](https://www.abhik.ai/concepts/category/deep-learning) (37) Neural network fundamentals: normalization, convolutions, losses, graph networks, and training dynamics. - [Calinski-Harabasz Index: The Variance Ratio Criterion](https://www.abhik.ai/concepts/deep-learning/calinski-harabasz): How the Calinski-Harabasz index evaluates clustering quality by measuring the ratio of between-cluster to within-cluster variance — fast, intuitive, and ideal for k-selection with convex clusters. - [Silhouette Score: Per-Point Clustering Evaluation](https://www.abhik.ai/concepts/deep-learning/silhouette-score): How the silhouette score measures clustering quality for every individual point — comparing intra-cluster cohesion to nearest-cluster separation, with per-point diagnostics and explicit distance-metric assumptions. - [Davies-Bouldin Index: Worst-Case Cluster Similarity](https://www.abhik.ai/concepts/deep-learning/davies-bouldin): How the Davies-Bouldin index evaluates clustering quality by finding each cluster's most similar neighbor — a pessimistic, worst-case metric that catches overlapping cluster pairs. - [Representation Collapse in Self-Supervised Learning](https://www.abhik.ai/concepts/deep-learning/collapse-risk): Understanding complete, dimensional, and cluster collapse — the failure modes that every self-supervised method must prevent. Learn why collapse happens and how contrastive, asymmetric, regularization, and masking approaches solve it. - [Batch Norm vs Layer Norm: When to Use Which](https://www.abhik.ai/concepts/deep-learning/batch-vs-layer-norm): BatchNorm normalizes over the batch and spatial axes; LayerNorm normalizes over the channel and spatial axes for each sample. The choice changes whether your model trains stably with batch=1, depends on batch composition at inference, and behaves consistently across train and eval. - [Convolution Operation: The Foundation of CNNs](https://www.abhik.ai/concepts/deep-learning/convolution-operation): Interactive guide to convolution in CNNs: visualize sliding windows, kernels, stride, padding, and feature detection with step-by-step demos. - [Cross-Entropy Loss for Classification](https://www.abhik.ai/concepts/deep-learning/cross-entropy-loss): Understand cross-entropy loss for classification: interactive demos of binary and multi-class CE, the -log(p) curve, softmax gradients, and focal loss. - [Dilated Convolutions: Expanding Receptive Fields Efficiently](https://www.abhik.ai/concepts/deep-learning/dilated-convolutions): Understand dilated (atrous) convolutions: how dilation rates expand receptive fields without extra parameters and how to avoid gridding artifacts. - [Receptive Field in CNNs](https://www.abhik.ai/concepts/deep-learning/receptive-field): Understand receptive fields in CNNs: how convolutional layers expand their field of view and the gap between theoretical and effective receptive fields. - [VAE Latent Space: Understanding Variational Autoencoders](https://www.abhik.ai/concepts/deep-learning/vae-latent-space): Explore VAE latent space in deep learning. Learn variational autoencoder encoding, decoding, interpolation, and the reparameterization trick. - [Contrastive Loss for Representation Learning](https://www.abhik.ai/concepts/deep-learning/contrastive-loss): Understand contrastive loss for representation learning: interactive demos of InfoNCE, triplet loss, and embedding space clustering with temperature tuning. - [Dropout Regularization](https://www.abhik.ai/concepts/deep-learning/dropout): Understand dropout regularization: how randomly silencing neurons prevents overfitting, the inverted dropout trick, and when to use each dropout variant. - [Focal Loss: Focusing on Hard Examples](https://www.abhik.ai/concepts/deep-learning/focal-loss): Learn focal loss for deep learning: down-weight easy examples, focus on hard ones. Interactive demos of gamma, alpha balancing, and RetinaNet. - [He/Kaiming Initialization](https://www.abhik.ai/concepts/deep-learning/he-initialization): Learn He (Kaiming) initialization for ReLU networks: why ReLU needs special weight initialization, variance flow, and dead neurons explained. - [KL Divergence in Machine Learning](https://www.abhik.ai/concepts/deep-learning/kl-divergence): Learn KL divergence for machine learning: measure distribution differences in VAEs, knowledge distillation, and variational inference. - [MSE and MAE Loss Functions](https://www.abhik.ai/concepts/deep-learning/mse-mae): Interactive guide to MSE vs MAE for regression: explore outlier sensitivity, gradient behavior, and Huber loss with visualizations. - [Xavier/Glorot Initialization](https://www.abhik.ai/concepts/deep-learning/xavier-initialization): Learn Xavier (Glorot) initialization: how it balances forward signals and backward gradients to enable stable deep network training with tanh and sigmoid. - [Adaptive Tiling: Efficient Visual Token Generation](https://www.abhik.ai/concepts/deep-learning/adaptive-tiling): Explore an illustrative region-tiling policy: resize tiles, calculate visual tokens, and distinguish analytical savings from measured model quality. - [Emergent Abilities in Large Language Models](https://www.abhik.ai/concepts/deep-learning/emergent-abilities): Explore emergent abilities in large language models: sudden capabilities at scale thresholds, phase transitions, and the mirage debate. - [Prompt Engineering for LLMs](https://www.abhik.ai/concepts/deep-learning/prompt-engineering): Master prompt engineering for large language models: from basic composition to Chain-of-Thought, few-shot, and advanced techniques. - [Prompt Information Flow in Causal Transformers](https://www.abhik.ai/concepts/deep-learning/prompt-influence-flow): Explore how prompt token positions connect through causal self-attention, and why connectivity and attention weights do not establish causal influence. - [Neural Scaling Laws Explained](https://www.abhik.ai/concepts/deep-learning/scaling-laws): Explore neural scaling laws in deep learning: power law relationships between model size, data, and compute that predict AI performance. - [Visual Complexity Analysis: Smart Image Processing](https://www.abhik.ai/concepts/deep-learning/visual-complexity-analysis): Learn visual complexity analysis in deep learning - how neural networks measure entropy, edges, and saliency for adaptive image processing. - [Gradient Flow in Deep Networks](https://www.abhik.ai/concepts/deep-learning/gradient-flow): Learn how gradients propagate through deep neural networks during backpropagation. Understand vanishing and exploding gradient problems. - [NAdam: Nesterov-Accelerated Adam](https://www.abhik.ai/concepts/deep-learning/nadam): Understand the NAdam optimizer that fuses Adam adaptive learning rates with Nesterov look-ahead momentum with a reproducible constant-momentum update and interactive optimization examples. - [Layer Normalization for Transformers](https://www.abhik.ai/concepts/deep-learning/layer-normalization): Learn layer normalization for transformers and sequence models: how normalizing across features enables batch-independent training. - [Internal Covariate Shift](https://www.abhik.ai/concepts/deep-learning/internal-covariate-shift): Understand internal covariate shift: why layer input distributions change during training, how it slows convergence, and how batch norm fixes it. - [Batch Normalization in Deep Learning](https://www.abhik.ai/concepts/deep-learning/batch-normalization): Learn batch normalization in deep learning: how normalizing layer inputs accelerates training, improves gradient flow, and acts as regularization. - [Skip Connections in Neural Networks](https://www.abhik.ai/concepts/deep-learning/skip-connections): Learn how skip connections and residual learning enable training of very deep neural networks. Understand the ResNet revolution with interactive visualizations. - [Graph Attention Networks (GAT)](https://www.abhik.ai/concepts/deep-learning/graph-attention-networks): Adaptive attention-based aggregation for graph neural networks - multi-head attention, learned weights, and interpretable graph learning - [Graph Centrality & Metrics](https://www.abhik.ai/concepts/deep-learning/graph-centrality): Understanding node importance through centrality measures, shortest paths, hop distances, clustering coefficients, and fundamental graph metrics - [Graph Convolutional Networks (GCN)](https://www.abhik.ai/concepts/deep-learning/graph-convolutional-networks): Learn Graph Convolutional Networks (GCN) with spectral theory, message passing, and node classification for geometric deep learning. - [Graph Embeddings and Node2Vec](https://www.abhik.ai/concepts/deep-learning/graph-embeddings): Learning low-dimensional vector representations of graphs through random walks, DeepWalk, Node2Vec, and skip-gram models - [Graph Pooling Methods](https://www.abhik.ai/concepts/deep-learning/graph-pooling): Hierarchical graph coarsening techniques - TopK, SAGPool, DiffPool, and readout operations for graph-level representations - [Self-Supervised Learning and Masked Prediction](https://www.abhik.ai/concepts/deep-learning/self-supervised-learning): Understand where self-supervised targets come from, how masked prediction creates a learning task, and why predicting pixels differs from predicting learned features. - [Target Encoders, EMA, and Stop-Gradient](https://www.abhik.ai/concepts/deep-learning/target-encoders-ema): Follow how a learned target encoder supplies feature targets, how stop-gradient separates backward paths, and how exponential moving averages update its parameters. - [Transfer Learning and Fine-Tuning](https://www.abhik.ai/concepts/deep-learning/transfer-learning-fine-tuning): Move a pretrained encoder to a downstream task: trace frozen features, head training, partial and full fine-tuning, learning rates, and BatchNorm state. ### [Computer Vision](https://www.abhik.ai/concepts/category/computer-vision) (28) Images, recognition, detection, segmentation, visual representations, camera geometry, and motion. - [Feature Pyramid Networks (FPN) Explained](https://www.abhik.ai/concepts/computer-vision/feature-pyramid-networks): Learn how Feature Pyramid Networks build multi-scale feature representations through top-down pathways and lateral connections for robust object detection. - [ASFF: Adaptive Spatial Feature Fusion](https://www.abhik.ai/concepts/computer-vision/asff): Learning where to fuse multi-scale features with per-pixel, per-level fusion weights. ASFF challenges FPN's uniform fusion assumption. - [RoI Pooling, RoI Align & Deformable RoI Pooling](https://www.abhik.ai/concepts/computer-vision/roi-pooling): Understanding region-based feature extraction for object detection, from quantized pooling to sub-pixel alignment and adaptive sampling - [Anchor-Based vs Anchor-Free Object Detection](https://www.abhik.ai/concepts/computer-vision/anchor-based-vs-anchor-free): Compare anchor-based vs anchor-free object detection: Faster R-CNN and RetinaNet anchors vs FCOS and CenterNet point-based methods. - [NAS-FPN: Learning to Design Feature Pyramid Networks](https://www.abhik.ai/concepts/computer-vision/nas-fpn): Understanding how neural architecture search discovers optimal feature pyramid architectures that outperform hand-designed alternatives - [DETR Explained: Object Detection with Transformers](https://www.abhik.ai/concepts/computer-vision/modern-object-detection): Understanding end-to-end object detection with transformers, from DETR's object queries to bipartite matching and attention-based localization - [NMS & Soft-NMS: Removing Duplicate Detections](https://www.abhik.ai/concepts/computer-vision/nms-soft-nms): Understanding Non-Maximum Suppression algorithms for object detection post-processing, from greedy NMS to soft variants - [Visual Complexity Analysis for Token Allocation](https://www.abhik.ai/concepts/computer-vision/visual-complexity-analysis): Compare edge, frequency and entropy heuristics for selecting a fixed budget of image patches, and distinguish token counts from accuracy and measured speed. - [Image Pixels, Channels, and Color Spaces](https://www.abhik.ai/concepts/computer-vision/image-pixels-color-spaces): Read an image as an array: inspect RGB channels, grayscale and HSV, and understand coordinates, data types and tensor layouts before preprocessing. - [Image Resizing and Interpolation](https://www.abhik.ai/concepts/computer-vision/image-resizing-interpolation): Compare nearest-neighbor and bilinear image resizing, see aliasing when downsampling, and understand aspect ratio, letterboxing and mask interpolation. - [Image Filtering and Edge Detection](https://www.abhik.ai/concepts/computer-vision/image-filtering-edge-detection): Inspect Gaussian smoothing and Sobel filters as local weighted sums, understand signed gradients and Canny edges, and connect fixed filters to CNNs. - [Image Classification and CNN Feature Maps](https://www.abhik.ai/concepts/computer-vision/image-classification-cnn): Follow an image through convolution, ReLU, pooling and class scores. Inspect a transparent toy CNN and learn what classification can and cannot predict. - [Bounding Boxes and Coordinate Systems](https://www.abhik.ai/concepts/computer-vision/bounding-boxes): Convert between xyxy, xywh and center-size boxes, normalize coordinates correctly, and avoid mistakes when resizing, clipping or flipping detection labels. - [How Object Detection Works](https://www.abhik.ai/concepts/computer-vision/object-detection): Understand object detection from image features to boxes, classes and scores. Compare one-stage and two-stage pipelines, training and duplicate handling. - [Intersection over Union (IoU)](https://www.abhik.ai/concepts/computer-vision/intersection-over-union): Calculate box intersection and union, explore how position and size change IoU, and distinguish overlap used for evaluation, NMS and localization losses. - [Object Detection Metrics: Precision, Recall, AP, and mAP](https://www.abhik.ai/concepts/computer-vision/object-detection-metrics): Match detections to objects, inspect precision–recall curves, calculate average precision, and distinguish confidence thresholds from IoU thresholds. - [Semantic, Instance, and Panoptic Segmentation](https://www.abhik.ai/concepts/computer-vision/image-segmentation): Compare semantic, instance and panoptic label maps, inspect individual pixels, and understand dense predictions, mask IoU, Dice and boundary errors. - [Image Augmentation and Label Consistency](https://www.abhik.ai/concepts/computer-vision/image-augmentation): Transform images and annotations together, inspect crop and flip geometry, and choose augmentation assumptions that preserve the meaning of the task. - [Visual Representations and Image Encoders](https://www.abhik.ai/concepts/computer-vision/visual-representations): Follow pixels into image features, compare spatial maps with pooled vectors, and see why image encoders expose different outputs for recognition and localization. - [Image Patches and Patch Embeddings](https://www.abhik.ai/concepts/computer-vision/image-patch-embeddings): Turn an image into a sequence: count patches, flatten pixels, apply a shared learned projection, and separate token count from embedding width. - [Object Queries in DETR](https://www.abhik.ai/concepts/computer-vision/object-queries): Follow spatial image features into DETR decoder slots, separating learned query embeddings, attention queries, changing content and object predictions. - [Set Prediction and Bipartite Matching](https://www.abhik.ai/concepts/computer-vision/bipartite-matching): Learn how DETR assigns unordered object targets to prediction slots, why greedy choices can fail, and how matching differs from training loss and evaluation. - [Bounding-Box Regression and Generalized IoU](https://www.abhik.ai/concepts/computer-vision/bounding-box-regression): Learn how normalized box coordinates, L1 loss, IoU, and Generalized IoU guide object localization, including disjoint boxes and containment limits. - [Evaluating Visual Representations](https://www.abhik.ai/concepts/computer-vision/evaluating-visual-representations): Evaluate frozen image features with nearest neighbors and linear probes, separate training from validation and test data, and understand what each protocol can establish. - [Dense Prediction and Upsampling](https://www.abhik.ai/concepts/computer-vision/dense-prediction-upsampling): Follow coarse feature maps into aligned pixel predictions: nearest and bilinear interpolation, transposed convolution, output shapes, skip fusion, and resizing logits before selecting labels. - [Deformable Attention and Reference Points](https://www.abhik.ai/concepts/computer-vision/deformable-attention): Follow a query from a spatial reference through learned offsets, bilinear feature sampling and weighted aggregation, with exact non-square coordinate arithmetic. - [Camera Geometry and Projection](https://www.abhik.ai/concepts/computer-vision/camera-geometry): Follow a 3D world point through camera extrinsics, depth division and pixel intrinsics, then see why one image point cannot determine its depth. - [Optical Flow and Visual Correspondence](https://www.abhik.ai/concepts/computer-vision/optical-flow): Follow image points between frames, distinguish forward flow from backward sampling, and understand brightness constancy, occlusion and the aperture problem. ### [Embeddings & Retrieval](https://www.abhik.ai/concepts/category/embeddings) (19) Dense and sparse embeddings, quantization, and vector search for semantic retrieval. - [Dense Embeddings](https://www.abhik.ai/concepts/embeddings/dense-embeddings): How dense embeddings turn meaning into geometry: word2vec, GloVe, and contextual models, vector arithmetic, cosine similarity, and where the field is heading. - [Sparse vs Dense vs Hybrid Retrieval: BM25, BERT, and Reranking Compared](https://www.abhik.ai/concepts/embeddings/sparse-vs-dense): How sparse retrieval (BM25/TF-IDF), dense retrieval (BERT-style embeddings), and hybrid systems that combine both compare on recall, semantic understanding, computational cost, and operational complexity for modern search. - [BM25 Algorithm for Text Retrieval](https://www.abhik.ai/concepts/embeddings/bm25-algorithm): Master the BM25 algorithm, the probabilistic ranking function powering Elasticsearch and Lucene for keyword-based document retrieval and search systems. - [Pooling Strategies](https://www.abhik.ai/concepts/embeddings/pooling-strategies): How a transformer’s per-token outputs become one embedding: CLS, mean, max, last-token, and attention pooling — what each does and when to use it. - [Contrastive Learning](https://www.abhik.ai/concepts/embeddings/contrastive-learning): Master contrastive learning for vector embeddings: how InfoNCE loss and self-supervised techniques train models to create high-quality semantic representations. - [Matryoshka Embeddings](https://www.abhik.ai/concepts/embeddings/matryoshka-embeddings): Matryoshka embeddings: nested representations enabling dimension reduction by simple truncation without model retraining for flexible retrieval. - [Domain Adaptation for Embeddings](https://www.abhik.ai/concepts/embeddings/domain-adaptation): Domain adaptation for embeddings: transfer learning to fine-tune retrieval models across domains while preventing catastrophic forgetting. - [Cross-Lingual Alignment](https://www.abhik.ai/concepts/embeddings/cross-lingual-alignment): Learn cross-lingual embedding alignment techniques like VecMap and MUSE for multilingual vector retrieval and zero-shot language transfer in search systems. - [Cross-Encoder vs Bi-Encoder](https://www.abhik.ai/concepts/embeddings/cross-encoder-vs-bi-encoder): Understand the fundamental differences between independent and joint encoding architectures for neural retrieval systems. - [Multi-Vector Late Interaction](https://www.abhik.ai/concepts/embeddings/multi-vector-late-interaction): Explore ColBERT and other multi-vector retrieval models that use fine-grained token-level matching for superior search quality. - [Hybrid Retrieval Systems](https://www.abhik.ai/concepts/embeddings/hybrid-retrieval-systems): Build hybrid retrieval systems combining BM25 sparse search with dense vector embeddings using reciprocal rank fusion for superior semantic search performance. - [Quantization Effects Simulator](https://www.abhik.ai/concepts/embeddings/quantization-effects): Embedding quantization simulator: explore memory-accuracy trade-offs from float32 to int8 and binary representations for retrieval. - [Vector Quantization Techniques](https://www.abhik.ai/concepts/embeddings/vector-quantization): Master vector compression techniques from scalar to product quantization. Learn how to reduce memory usage by 10-100× while preserving search quality. - [Binary Embeddings for Fast Search](https://www.abhik.ai/concepts/embeddings/binary-embeddings): Learn how binary embeddings use 1-bit quantization for ultra-compact vector representations, enabling billion-scale similarity search with 32x memory reduction. - [Vector Index Structures](https://www.abhik.ai/concepts/embeddings/index-structures): Explore the fundamental data structures powering vector databases: trees, graphs, hash tables, and hybrid approaches for efficient similarity search. - [HNSW vs IVF-PQ vs LSH: Approximate Nearest Neighbor Algorithms Compared](https://www.abhik.ai/concepts/embeddings/ann-comparison): How HNSW, IVF-PQ, and LSH compare for approximate nearest neighbor (ANN) search — recall, latency, memory, build cost, and update characteristics — with Annoy, ScaNN, and DiskANN included for completeness. - [LSH: Locality Sensitive Hashing](https://www.abhik.ai/concepts/embeddings/lsh-search): Explore how LSH uses probabilistic hash functions to find similar vectors in sub-linear time, perfect for streaming and high-dimensional data. - [IVF-PQ: Inverted File with Product Quantization](https://www.abhik.ai/concepts/embeddings/ivf-pq): Learn how IVF-PQ combines clustering and compression to enable billion-scale vector search with minimal memory footprint. - [HNSW: Hierarchical Navigable Small World](https://www.abhik.ai/concepts/embeddings/hnsw-search): How HNSW navigates a layered proximity graph to find nearest neighbors in logarithmic time — the default in-memory index of modern vector databases. ### [GPU & High-Performance Computing](https://www.abhik.ai/concepts/category/gpu-computing) (28) CUDA, tensor cores, multi-GPU communication, and cluster-scale workload orchestration. - [Slurm Fundamentals: Job Scheduling on HPC Clusters](https://www.abhik.ai/concepts/gpu-computing/slurm-fundamentals): Complete guide to Slurm — architecture, core commands, job lifecycle, job scripts, array jobs, dependencies, monitoring with squeue/sacct, and troubleshooting failed jobs on HPC clusters. - [Tensor Cores: Mixed Precision & Matrix Acceleration](https://www.abhik.ai/concepts/gpu-computing/tensor-cores): How NVIDIA Tensor Cores accelerate GEMM: warp-level MMA, precision dials (TF32/FP16/BF16/INT8/FP8), shape alignment and padding cost, AMP vs model.half(), and when the speedup actually shows up. - [High Bandwidth Memory (HBM)](https://www.abhik.ai/concepts/gpu-computing/hbm-memory): How HBM feeds AI GPUs: the memory wall, width-over-speed packaging, TSVs and interposers, the on-package hierarchy, and the roofline that decides if your kernel is bandwidth-bound. - [GPU Memory Hierarchy & Optimization](https://www.abhik.ai/concepts/gpu-computing/memory-hierarchy): Master GPU memory hierarchy from registers to global memory, understand coalescing patterns, bank conflicts, and optimization strategies for maximum performance - [Multi-GPU Communication: NVLink, PCIe, and NCCL](https://www.abhik.ai/concepts/gpu-computing/multi-gpu-communication): How GPUs talk: the bandwidth cliff from HBM to Ethernet, NVLink 5 and GB200 NVL72 topologies, ring AllReduce step by step, and choosing between NCCL, Gloo, and MPI. - [GPU Streaming Multiprocessor (SM)](https://www.abhik.ai/concepts/gpu-computing/shared-multiprocessor): How a single SM actually runs work: warps of 32, cutaway of cores and memory, divergence tax, latency-hiding occupancy, and coalesced loads — instruments, not a catalog. - [Slurm GPU Allocation for Distributed Training](https://www.abhik.ai/concepts/gpu-computing/slurm-gpu-allocation): Complete guide to GPU allocation on Slurm — --gres flags, CUDA_VISIBLE_DEVICES remapping, GPU topology and NVLink binding, MIG partitioning, production job scripts, and debugging common GPU errors. - [CUDA Contexts: Ownership, Current Stack, Isolation](https://www.abhik.ai/concepts/gpu-computing/cuda-context): Deep dive into the CUDA context object: control vs data plane, inventory (memory, modules, streams, events, graphs), push/pop/setCurrent stacks, primary retain/release, flags and limits, isolation, cost, and traps. - [CUDA Streams: Asynchronous Execution and Concurrency](https://www.abhik.ai/concepts/gpu-computing/cuda-streams): A CUDA stream is an in-order queue of GPU ops. Overlap H→D, kernels, and D→H across streams — and avoid the legacy default-stream fences between blocking streams in one context. - [Slurm Resource Management and Job Priority](https://www.abhik.ai/concepts/gpu-computing/slurm-resource-management): How Slurm decides which jobs run first — priority factors, fair-share scheduling, backfill, and monitoring commands (squeue, sinfo, sacct). - [NVIDIA Unified Virtual Memory](https://www.abhik.ai/concepts/gpu-computing/unified-memory): NVIDIA Unified Virtual Memory (UVM): on-demand page migration, memory oversubscription, and simplified CPU-GPU memory management. - [CUDA Context vs Streams vs MPS: Which Layer Fixes What](https://www.abhik.ai/concepts/gpu-computing/cuda-context-vs-streams): Decision map for CUDA: a context is per-process GPU state, a stream is an in-order queue inside a context, and MPS co-schedules work across client contexts on Volta+. Pick the layer that matches the problem. - [Flynn's Classification: Taxonomy of Computer Architectures](https://www.abhik.ai/concepts/gpu-computing/flynns-classification): Flynn's Classification explained — SISD, SIMD, MISD, MIMD with interactive architecture explorer, SIMD evolution from MMX to AMX, branch divergence visualization, and workload-architecture throughput comparison. - [MPI Fundamentals: Message Passing for Distributed Computing](https://www.abhik.ai/concepts/gpu-computing/mpi-fundamentals): Complete MPI guide — point-to-point and collective communication with real C and mpi4py code, deadlock simulation, analytical performance modeling, communicator splitting, and debugging on HPC clusters. - [OpenMP: Shared-Memory Parallel Programming](https://www.abhik.ai/concepts/gpu-computing/openmp): OpenMP parallel programming: fork-join model, scheduling, data races, false sharing, NUMA thread affinity, and GPU offloading. - [Page Migration & Fault Handling](https://www.abhik.ai/concepts/gpu-computing/page-migration): CUDA page migration and fault handling between CPU and GPU memory. Learn TLB management, DMA transfers, and memory optimization. - [Slurm Accounting and Resource Tracking](https://www.abhik.ai/concepts/gpu-computing/slurm-accounting): How Slurm tracks resource consumption through account hierarchies, TRES billing, and resource limits — sacctmgr, sreport, and the association model explained. - [CUDA Multi-Process Service (MPS): Concurrent Client Contexts](https://www.abhik.ai/concepts/gpu-computing/cuda-mps): Why ordinary multi-process CUDA time slicing can leave SMs idle, how Volta+ MPS co-schedules separate client contexts, execution-resource caps, and when to choose MPS or MIG. - [HPC Performance Optimization: Scaling, Profiling, and Tuning](https://www.abhik.ai/concepts/gpu-computing/hpc-performance-optimization): Amdahl and Gustafson ceilings, strong vs weak scaling, roofline bounds, and hiding all-reduce behind compute — the levers that decide whether more GPUs actually buy science. - [Slurm Backfill Scheduling: How Small Jobs Fill the Gaps](https://www.abhik.ai/concepts/gpu-computing/slurm-backfill): How sched/backfill works — the algorithm that lets small jobs run in gaps while large jobs wait, why accurate time limits matter, and the key tuning parameters (bf_interval, bf_window, bf_max_job_test). - [NVIDIA vs AMD for Deep Learning: CUDA vs ROCm and the Datacenter Accelerators](https://www.abhik.ai/concepts/gpu-computing/nvidia-vs-amd): NVIDIA vs AMD for deep learning compared at both layers: the CUDA vs ROCm software moat, the microarchitecture (warp vs wavefront, SM vs CU, Tensor vs Matrix Cores), and the datacenter accelerators (H100/H200/B200 vs MI300X/MI325X). - [NVIDIA Device Files in /dev/](https://www.abhik.ai/concepts/gpu-computing/nvidia-device-files): How /dev/nvidia*, nvidiactl, nvidia-uvm, and DRI nodes map major/minor numbers to the driver, what CUDA opens first, and the minimum mount set for containers. - [NVIDIA GPU Operator: Declarative GPU Stacks on Kubernetes](https://www.abhik.ai/concepts/gpu-computing/kubernetes-operator): How the NVIDIA GPU Operator turns drivers, container toolkit, device plugin, GFD, DCGM, and MIG into DaemonSets reconciling from a ClusterPolicy — boot order, request path, sharing modes, and ops traps. - [NVIDIA Persistence Daemon: Killing GPU Cold Starts](https://www.abhik.ai/concepts/gpu-computing/nvidia-persistence-daemon): Why the first nvidia-smi or CUDA open costs seconds, what that cold path rebuilds, how nvidia-persistenced holds device FDs on the host, and when persistence pays off for pods, CI, and batch jobs. - [Distributed Parallelism in Deep Learning](https://www.abhik.ai/concepts/gpu-computing/distributed-parallelism): GPU distributed parallelism: Data Parallel (DDP), Tensor Parallel, Pipeline Parallel, and ZeRO optimization for training large AI models. - [GPU Pipeline Hazards: Scoreboard Stalls and Warp Divergence](https://www.abhik.ai/concepts/gpu-computing/gpu-pipeline-hazards): A fine kernel can still profile poorly: warp divergence taxes throughput ~1/N; alone on a long scoreboard wait, SM issue util can hit 0%. Interactive demos. - [NCCL: How NVIDIA Collective Communication Works](https://www.abhik.ai/concepts/gpu-computing/nccl-communication): A deep dive into NCCL internals: communicators and channels, how it picks ring/tree/NVLS algorithms and LL/LL128/Simple protocols, reading NCCL_DEBUG logs, and tuning and debugging distributed training. - [Data vs Tensor vs Pipeline Parallelism: When to Use Each](https://www.abhik.ai/concepts/gpu-computing/data-vs-tensor-vs-pipeline-parallelism): Data, tensor and pipeline parallelism split the batch, weight matrices or layers across GPUs. Compare memory, traffic and batch limits, then combine them. ### [Systems & Architecture](https://www.abhik.ai/concepts/category/systems) (49) CPU pipelines, memory hierarchies, the Linux kernel, filesystems, and networking. - [CPU Cache Lines: The Complete Guide with Interactive Simulator](https://www.abhik.ai/concepts/systems/cpu-cache-lines): Deep dive into CPU cache lines — interactive cache simulator with configurable associativity and replacement policies, false sharing MESI protocol visualization, access pattern benchmarks, and optimization techniques. - [How RAM Works: Interactive Deep Dive into Computer Memory](https://www.abhik.ai/concepts/systems/how-ram-works): Explore the inner workings of RAM through beautiful animations and interactive visualizations. Understand memory cells, addressing, and the memory hierarchy. - [initramfs: The Initial RAM Filesystem Explained](https://www.abhik.ai/concepts/systems/initramfs-boot-process): Learn how initramfs enables Linux boot by loading essential drivers before the root filesystem mounts. Explore early userspace initialization. - [Linux Kernel Architecture: From User Space to Hardware](https://www.abhik.ai/concepts/systems/kernel-architecture): Explore Linux kernel architecture: user and kernel space, cooperating subsystems, cached and uncached file reads, and how containers share one kernel. - [Filesystems: The Digital DNA of Data Storage](https://www.abhik.ai/concepts/systems/filesystems-overview): Explore Linux filesystems through interactive visuals. Learn VFS, compare ext4 vs Btrfs vs ZFS, and understand file operations. - [Virtual Memory & TLB: Complete Guide to Address Translation](https://www.abhik.ai/concepts/systems/virtual-memory): Master virtual memory and TLB address translation with interactive demos. Learn page tables, page faults, and memory management optimization. - [Filesystem Journaling: Write-Ahead Logging](https://www.abhik.ai/concepts/systems/filesystem-journaling): Follow a filesystem transaction from journal to checkpoint. Explore crash recovery, ext4 journal modes, and the limits of data durability. - [Inodes: The Hidden Metadata That Powers Every File](https://www.abhik.ai/concepts/systems/inodes): Understand Linux inodes - the metadata structures behind every file. Learn about hard links, soft links, and inode limits. - [Memory Access Patterns: Sequential vs Strided](https://www.abhik.ai/concepts/systems/memory-access-patterns): Master sequential vs strided memory access patterns. Learn how cache efficiency and hardware prefetching affect application performance. - [Copy-on-Write (CoW): Blocks, Snapshots, and Atomic Commits](https://www.abhik.ai/concepts/systems/copy-on-write): Follow filesystem copy-on-write from a changed block to a committed root. Explore snapshot sharing, space reclamation, and the trade-offs in Btrfs and ZFS. - [FUSE: Filesystem in Userspace Explained](https://www.abhik.ai/concepts/systems/fuse-filesystem): Learn FUSE (Filesystem in Userspace) for building custom filesystems. Understand how NTFS-3G, SSHFS, and cloud storage work. - [ext4: The Linux Workhorse Filesystem](https://www.abhik.ai/concepts/systems/ext4-filesystem): Explore ext4, the default Linux filesystem with journaling, extents, and proven reliability. Learn how ext4 protects your data. - [CPU Pipeline Architecture](https://www.abhik.ai/concepts/systems/cpu-pipeline-detailed): Follow instructions through a five-stage pipeline, distinguish latency from throughput, and connect issue width to out-of-order execution. - [Hazard Detection: Pipeline Dependencies and Solutions](https://www.abhik.ai/concepts/systems/hazard-detection): Trace data, structural and control hazards, understand forwarding and load-use stalls, and connect register dependencies to scheduling. - [Branch Prediction & Speculation](https://www.abhik.ai/concepts/systems/cpu-pipelines): Replay branch outcomes to compare predictors, then connect local and global history to speculative execution, recovery and profiling. - [Mount Options: Filesystem Behavior and Performance](https://www.abhik.ai/concepts/systems/mount-options): Master Linux mount options like noatime and async for performance tuning and security hardening. Interactive guide to fstab configuration. - [NTFS Filesystem: The Master File Table](https://www.abhik.ai/concepts/systems/ntfs-filesystem): NTFS internals from the Master File Table outward: 1 KB attribute records, resident vs non-resident $DATA, run lists, alternate data streams, the $LogFile journal, and why dual-boot Linux distros prefer ntfs3 over ntfs-3g. - [Btrfs: Modern Copy-on-Write Filesystem](https://www.abhik.ai/concepts/systems/btrfs-filesystem): Trace how Btrfs shares extents, creates snapshots, distributes copies, compresses writes, and repairs checksum errors when a good replica exists. - [Filesystem Data Integrity: Detecting Silent Corruption](https://www.abhik.ai/concepts/systems/filesystem-integrity): Explore checksum verification, Merkle-tree reads, mirror repair, and scrubbing through interactive examples of silent corruption and recovery. - [ZFS: The Ultimate Filesystem](https://www.abhik.ai/concepts/systems/zfs-filesystem): Master ZFS filesystem with pooled storage, RAID-Z, snapshots, and checksums. Learn enterprise-grade data integrity on Linux. - [XFS: High-Performance Parallel Filesystem](https://www.abhik.ai/concepts/systems/xfs-filesystem): XFS internals end-to-end: allocation groups for lock-free parallel metadata, B+ trees instead of bitmaps, extent-based allocation that scales to terabytes, and delayed allocation that turns scattered writes into contiguous extents. - [FAT32 & exFAT: Universal Filesystems](https://www.abhik.ai/concepts/systems/fat-filesystems): Learn FAT32 and exFAT filesystems for cross-platform USB drives and SD cards. Understand file size limits and compatibility. - [Memory Controllers: The Brain Behind RAM Management](https://www.abhik.ai/concepts/systems/memory-controllers): Learn how memory controllers manage CPU-RAM data flow. Interactive demos of channels, ranks, banks, and command scheduling for optimal bandwidth. - [Memory Interleaving: Parallel Memory Access](https://www.abhik.ai/concepts/systems/memory-interleaving): Discover how memory interleaving distributes addresses across banks for parallel access. Boost memory bandwidth in DDR5 and GPU systems. - [NUMA Architecture: Non-Uniform Memory Access](https://www.abhik.ai/concepts/systems/numa-architecture): Explore NUMA architecture and memory locality in multi-socket systems. Understand local vs remote memory access latency and optimization strategies. - [RAID: Redundant Arrays for Speed and Safety](https://www.abhik.ai/concepts/systems/raid-storage): RAID storage visualized: RAID 0, 1, 5, 6, and 10 levels explained. Learn how they work, when to use them, and disk failure recovery. - [SoA vs AoS: Data Layout Optimization](https://www.abhik.ai/concepts/systems/soa-vs-aos): Structure of Arrays vs Array of Structures as instruments: cache-line fill, SIMD gather vs contiguous load, GPU coalescing, and AoSoA hybrids — when layout is a 10× decision. - [Linux Process Management: Fork, Exec, and Beyond](https://www.abhik.ai/concepts/systems/process-management): Master Linux process management through interactive visualizations. Understand process lifecycle, fork/exec operations, zombies, orphans, and CPU scheduling. - [Linux Memory Management: Virtual Memory, Paging, and Beyond](https://www.abhik.ai/concepts/systems/memory-management): Explore Linux memory management through interactive visualizations. Understand virtual memory, page tables, TLB, swapping, and memory allocation. - [Transparent Huge Pages (THP): Reducing TLB Pressure](https://www.abhik.ai/concepts/systems/transparent-huge-pages): Learn how Transparent Huge Pages (THP) reduces TLB misses by promoting 4KB to 2MB pages. Understand performance benefits and memory bloat tradeoffs. - [Linux System Calls: The User-Kernel Interface](https://www.abhik.ai/concepts/systems/system-calls): Linux system calls visualized: how user programs communicate with the kernel, protection rings, argument registers, error handling, and syscall performance. - [Client-Server Communication: Polling vs WebSockets](https://www.abhik.ai/concepts/systems/client-server-communication): Learn client-server communication patterns including short polling, long polling, and WebSockets. Compare HTTP protocols for real-time web applications. - [Long Polling: The Patient Connection](https://www.abhik.ai/concepts/systems/long-polling): Learn HTTP long polling - a server-side technique that holds connections open until data arrives. Achieve near real-time updates with standard protocols. - [Linux Networking Stack: From Packets to Applications](https://www.abhik.ai/concepts/systems/networking-stack): Master the Linux networking stack through interactive visualizations. Understand TCP/IP layers, sockets, iptables, routing, and network namespaces. - [Short Polling: The Impatient Client Pattern](https://www.abhik.ai/concepts/systems/short-polling): Learn short polling in networking - a simple HTTP pattern for periodic data fetching. See why 70-90% of requests waste bandwidth and when to use alternatives. - [Understanding TCP/IP Protocol Stack](https://www.abhik.ai/concepts/systems/tcp-ip): Explore the TCP/IP protocol stack, packet encapsulation, and how data travels through network layers from application to physical transmission. - [WebSockets: Real-Time Bidirectional Communication](https://www.abhik.ai/concepts/systems/websocket): Master WebSocket protocol for real-time bidirectional communication over TCP. Learn handshakes, frames, and building low-latency web applications. - [Linux Boot Process: From Power-On to Login](https://www.abhik.ai/concepts/systems/boot-process): Visualize the complete Linux boot sequence from BIOS/UEFI to login. Learn how GRUB, kernel, and systemd work together with interactive visualizations. - [Linux Init Systems: From SysV to systemd](https://www.abhik.ai/concepts/systems/init-systems): Compare Linux init systems through interactive visualizations. Understand the evolution from SysV Init to systemd, service management, and boot orchestration. - [Linux Kernel Modules: Extending the Kernel at Runtime](https://www.abhik.ai/concepts/systems/kernel-modules): Master Linux kernel modules through interactive visualizations. Learn how to load, unload, develop, and debug kernel modules that extend Linux functionality. - [Linux Namespaces: The Foundation of Container Isolation](https://www.abhik.ai/concepts/systems/namespaces): Master Linux namespaces — the kernel mechanism that makes containers possible. Learn how mount, PID, network, and user namespaces create isolated environments, with interactive demos. - [Wayland vs X11: Modern Display Server Architecture](https://www.abhik.ai/concepts/systems/wayland-x11): Compare Wayland vs X11 display servers on Linux. Learn about architecture, performance, security, and modern graphics stack. - [Linux cgroups: Resource Limits for Processes](https://www.abhik.ai/concepts/systems/cgroups): Control groups as instruments: noisy neighbors, v1 vs v2 membership, CFS cpu.max periods, memory.high vs memory.max OOM, and how Docker flags become cgroup files. - [Containers Under the Hood: From Primitives to Docker](https://www.abhik.ai/concepts/systems/containers): Discover how containers work by combining namespaces, cgroups, and OverlayFS. Build a mental model of Docker internals through interactive visualizations. - [How Docker Works with GPUs: Device Files, Bind Mounts, and Driver Stacks](https://www.abhik.ai/concepts/systems/gpu-containers): Understand how containerized processes access GPU hardware through device files, bind mounts, and the NVIDIA container runtime. Learn the kernel driver vs user-space library distinction. - [nvidia-modeset: Kernel Mode-Setting for NVIDIA GPUs](https://www.abhik.ai/concepts/systems/nvidia-modeset): Learn nvidia-modeset for display configuration on Linux. Understand kernel mode-setting, DRM integration, and GPU drivers. - [Dennard Scaling: From Free Speed to the Power Wall](https://www.abhik.ai/concepts/systems/dennard-scaling): Explore how smaller transistors promised more speed at the same power, why voltage limits broke that bargain, and how chip design adapted. - [NAT: How Many Devices Share One Public Address](https://www.abhik.ai/concepts/systems/nat): Step a packet through a home router to see how NAT rewrites addresses and ports, what it breaks, how hole punching and NAT-T work around it, and why it is not security. - [VPN: How a Tunnel Changes the Address Websites See](https://www.abhik.ai/concepts/systems/vpn): Follow a packet inside an encrypted VPN tunnel, separate wrapping from NAT, compare full and split routing, and see what Wi-Fi, ISPs and VPN servers can observe. ### [Language & Framework Internals](https://www.abhik.ai/concepts/category/language-internals) (33) How CPython, C++, and PyTorch work under the hood: bytecode, memory, linking, and concurrency. - [Python Bytecode Compilation](https://www.abhik.ai/concepts/language-internals/python-bytecode-compilation): Explore CPython bytecode compilation from source to .pyc files. Learn the dis module, PVM stack operations, and Python 3.11+ adaptive specialization. - [PyTorch DataLoader Pipeline](https://www.abhik.ai/concepts/language-internals/pytorch-dataloader-pipeline): Follow PyTorch batches through worker queues, collation, ordered delivery and GPU transfer, then measure loading throughput and budget process memory. - [C++ Stack vs Heap Memory: A Complete Guide](https://www.abhik.ai/concepts/language-internals/cpp-stack-vs-heap): Deep dive into C++ memory allocation — stack frame internals, heap allocator mechanics, fragmentation, performance benchmarks, custom allocators, RAII, and debugging with AddressSanitizer and Valgrind. - [Python Memory Management](https://www.abhik.ai/concepts/language-internals/python-memory-management): A practical mental model for CPython memory management: names and references, object headers, PyMalloc arenas, reference counting, reuse paths, and memory profiling. - [Python Global Interpreter Lock (GIL)](https://www.abhik.ai/concepts/language-internals/global-interpreter-lock): Follow CPython thread states, execution frames and shared objects through a GIL handoff. Inspect the source behind waiting, release, native work and application locks. - [Pinned Memory and DMA Transfers in PyTorch](https://www.abhik.ai/concepts/language-internals/pytorch-pin-memory): How page-locked memory changes CPU-to-GPU transfers, the extra staging in pageable paths, and the conditions needed to overlap copies with GPU work. - [C++ Symbol Resolution: How the Linker Connects Your Code](https://www.abhik.ai/concepts/language-internals/cpp-symbol-resolution): Complete guide to C++ symbol resolution — how linkers match references to definitions, name mangling, strong vs weak symbols, ODR, template instantiation, linking order, and debugging undefined reference errors. - [C++ Program Loading: From ELF to Running Process](https://www.abhik.ai/concepts/language-internals/cpp-program-loading): How Linux loads C++ programs: captured ELF segments, runtime mappings, startup handoffs, lazy binding and practical loader debugging. - [Understanding num_workers](https://www.abhik.ai/concepts/language-internals/pytorch-dataloader-num-workers): Why DataLoader num_workers matters: processes hide load latency behind GPU work, how to find the sweet spot, and the memory/GIL pitfalls that come with the pool. - [Python Object Model Internals](https://www.abhik.ai/concepts/language-internals/python-object-model): Learn how CPython implements PyObject, type objects, and the unified object model. Explore reference counting, memory layout, and Python internals. - [Python Garbage Collection](https://www.abhik.ai/concepts/language-internals/python-garbage-collection): Understand CPython garbage collection: reference counting, generational GC for circular references, weak references, and gc module tuning strategies. - [Python __slots__ Optimization](https://www.abhik.ai/concepts/language-internals/python-slots): Learn when Python __slots__ reduces memory, how slot storage differs from __dict__, and the caveats for dataclasses and inheritance. - [Python Green Threads vs OS Threads](https://www.abhik.ai/concepts/language-internals/green-threads-vs-os-threads): Complete guide to Python concurrency — OS threads, green threads (asyncio), the GIL, event loop internals, Python 3.13 free-threading, and production patterns. - [Python asyncio Event Loop](https://www.abhik.ai/concepts/language-internals/asyncio-event-loop): Deep dive into Python's asyncio library, understanding event loops, coroutines, tasks, and async/await patterns with interactive visualizations. - [Python Shared Memory](https://www.abhik.ai/concepts/language-internals/python-shared-memory): Master Python multiprocessing.shared_memory for zero-copy IPC. Learn synchronization, NumPy integration, and race condition prevention patterns. - [Thread Safety: Concurrent Programming Fundamentals](https://www.abhik.ai/concepts/language-internals/cpp-thread-safety): Complete C++ thread safety guide — race conditions with step-through simulation, mutexes, atomics, condition variables, deadlock detection, memory ordering, and Thread Sanitizer walkthrough. - [C++ AST & Parsing Explained](https://www.abhik.ai/concepts/language-internals/cpp-ast-parsing): Follow a C++ function from tokens to a simplified AST, then inspect exact source ranges, declaration references and types in a keyboard-accessible explorer. - [C++ Compilation Overview](https://www.abhik.ai/concepts/language-internals/cpp-compilation): Understand the complete C++ compilation pipeline from source code to object files. Learn preprocessing, parsing, code generation, and optimization stages. - [C++ Dynamic Linking at Runtime](https://www.abhik.ai/concepts/language-internals/dynamic-linking): Follow Linux shared libraries from dependency lookup to PLT/GOT binding, then build a versioned library, load a plugin and debug runtime linking failures. - [C++ Linking Overview](https://www.abhik.ai/concepts/language-internals/cpp-linking): How C++ object files are linked into executables. Learn symbol resolution, static vs dynamic linking, and linker optimization. - [Memory Management & RAII in C++](https://www.abhik.ai/concepts/language-internals/cpp-raii): Learn Resource Acquisition Is Initialization (RAII) - the cornerstone of C++ memory management. Understand automatic resource cleanup and exception safety. - [Object-Oriented Programming in C++](https://www.abhik.ai/concepts/language-internals/cpp-inheritance-polymorphism): Master C++ OOP concepts including inheritance, polymorphism, virtual functions, and modern object-oriented design principles with interactive examples. - [C++ Compiler Optimization](https://www.abhik.ai/concepts/language-internals/cpp-optimization): C++ compiler optimization lab notebook — compare optimization levels, inspect compiler rewrites, diagnose auto-vectorization, and build production verification commands. - [Pointers & References in C++](https://www.abhik.ai/concepts/language-internals/cpp-pointers-references): Understand C++ pointers and references with step-by-step memory diagrams: addressing, dereferencing, aliasing, array arithmetic, and object lifetime. - [C++ Preprocessor Directives](https://www.abhik.ai/concepts/language-internals/cpp-preprocessor): C++ preprocessor visualized: macros, header guards, conditional compilation, and #include directives explained interactively. - [Smart Pointers in Modern C++](https://www.abhik.ai/concepts/language-internals/cpp-smart-pointers): Master C++11 smart pointers through interactive examples. Learn unique_ptr, shared_ptr, and weak_ptr with reference counting visualizations. - [Templates & STL in C++](https://www.abhik.ai/concepts/language-internals/cpp-templates-stl): Master C++ templates and the Standard Template Library. Learn generic programming, template metaprogramming, and STL containers and algorithms. - [DataParallel vs DistributedDataParallel](https://www.abhik.ai/concepts/language-internals/pytorch-dataparallel-vs-ddp): Compare PyTorch DataParallel vs DistributedDataParallel for multi-GPU training. Learn GIL limitations, NCCL AllReduce, and DDP best practices. - [C++ Virtual Tables & Inheritance](https://www.abhik.ai/concepts/language-internals/cpp-virtual-tables): C++ virtual tables (vtables) explained. Learn virtual dispatch, single/multiple inheritance, RTTI, and object memory layout visually. - [Tail Call Optimization and Python’s Tail-Call Interpreter](https://www.abhik.ai/concepts/language-internals/tail-call-optimization): See how tail calls reuse stack frames, when compilers can eliminate them, and how CPython 3.14 applies tail calls to C opcode dispatch. - [Interpreter Dispatch: Switch, Computed Goto, and Tail Calls](https://www.abhik.ai/concepts/language-internals/interpreter-dispatch): Trace a small virtual machine through branches and loops, then compare switch, computed goto, and tail-call dispatch without changing its bytecode semantics. - [Continuation-Passing Style: Make the Remaining Work Explicit](https://www.abhik.ai/concepts/language-internals/continuation-passing-style): Follow direct-style and CPS factorial through the same calculation. See how closures carry pending work, why Python still grows call depth, and where trampolines fit. - [Trampolines: Recursive Control Flow Through a Loop](https://www.abhik.ai/concepts/language-internals/trampoline-recursion): Step through mutual recursion and a Call/Done trampoline to see deferred work, bounded worker depth, base cases, and the allocation tradeoff. ## Talks (7) All talks: https://www.abhik.ai/speaking - [GPU Programming 101 in Python](https://www.abhik.ai/talks/gpu-programming-101-in-python): A hands-on workshop introducing GPU kernel programming using Triton — write high-performance GPU kernels in pure Python without needing deep CUDA expertise. - [ArrPy: Array You Fast Enough?](https://www.abhik.ai/talks/arrpy-array-you-fast-enough): A hands-on workshop rebuilding NumPy from scratch — from pure Python loops through Cython to SIMD-optimized C++ via pybind11. - [Multimodal Models: History and How They Work](https://www.abhik.ai/talks/multimodal-models): A brief history of vision-language models, from CLIP's contrastive dual encoders to PaliGemma's image-prefix decoder, with interactive visualizations. - [Rolling with Python: A Deep Dive into Python Wheels](https://www.abhik.ai/talks/rolling-with-python-intro-to-python-wheels): An in-depth exploration of Python wheels, their importance in package distribution, and how they integrate C/C++ extensions. - [Speeding up Python with Cython: From Slow to Lightning Fast](https://www.abhik.ai/talks/speeding-up-python-with-cython): A comprehensive guide to using Cython to significantly improve the performance of Python code in compute-intensive applications. - [The Age of Digital Da Vinci: All About Image Generation](https://www.abhik.ai/talks/the-age-of-digital-da-vinci): A journey through the evolution of image generation technologies, from encoder-decoders to GANs and the revolutionary Stable Diffusion models. - [Introduction to Machine Learning](https://www.abhik.ai/talks/introduction-to-machine-learning): A beginner-friendly introduction to machine learning fundamentals, covering supervised and unsupervised learning with practical examples. ## Feeds & Discovery - RSS 2.0: https://www.abhik.ai/rss/feed.xml - Atom: https://www.abhik.ai/rss/atom.xml - JSON Feed: https://www.abhik.ai/rss/feed.json - Sitemap: https://www.abhik.ai/sitemap.xml ## Additional Resources - [GitHub Profile](https://github.com/abhiksark): Open source contributions and projects - [LinkedIn Profile](https://linkedin.com/in/abhiksark): Professional experience - [X (Twitter)](https://x.com/abhiksark): Technical updates and insights - [Substack](https://abhiksark.substack.com): Performance engineering, GPU computing and inference systems