GPU Pipeline Hazards: Scoreboard Stalls and Warp Divergence

The kernel is fine. The profile isn’t.

You’ve been here: the answers match the CPU, the tests are green, nothing is illegal. And yet the kernel is slow. Occupancy looks okay on paper. You’re not “out of FLOPs.” You open the profiler expecting to find a math bottleneck and get something blunter — the SM is waiting.

Two stories show up over and over.

Sometimes half the warp takes if and half takes else. You pictured free parallel SIMT. The machine runs one path, then the other. Idle lanes still sit there. Cost goes up; useful work per pass goes down.

Sometimes a warp issues a global load and needs the result for the next math. While it’s stuck, the SM has nobody else to run. Issue util drops to nothing until the data comes back. Same load with other ready warps? The wait barely shows up — someone else was always ready to issue.

Those are control and data hazards, GPU edition. The rest of this page is just those two stories with numbers you can quote and toys you can poke.

Two numbers

1. Divergence · throughput ≈ 1/N
If a warp splits into N paths, hardware serializes them. if/else → about half the useful work per pass (~50%). Four-way → ~25%. You already paid N× wall time for that branch region.

2. Scoreboard · alone on a wait → util ~0%
A warp parked on a long load cannot issue dependent work. Alone on the SM, issue util can sit near 0% for hundreds of cycles. With other ready warps, the same wait is latency hiding — the SM keeps busy.

Flip the demos below until both numbers feel obvious.

Control hazard: warp divergence

Each color is a path group. Hit serialize and watch them take turns. Switch 4-way or 8-way and the useful bar falls toward 1/N.

Fix: Prefer warp-uniform branches on hot paths. Put divergence across warps (or blocks), not inside a warp. On hot loops, prefer predication or a rewrite over data-dependent multi-way control when you can.

Data hazard: long scoreboard

Same load → use chain. The tape is SM issue each cycle. Flip 1 warp vs 3 warps: alone, the middle goes hollow (util 0% while W0 waits); with helpers, that gap fills in.

Fix: Break pure load→use chains so other warps (or independent work in the same warp) can issue. Raise useful occupancy for memory-bound kernels — not “max occupancy” as a religion.

Dictionary (CPU → GPU)

Same problem, two machines. Tap Control / Data / Structural — watch both sides move. The number in the middle is the GPU hit.

What to do

1. Profile stall reasons (Nsight Compute) before rewriting math.
2. Kill warp divergence on hot paths — or accept the 1/N tax deliberately.
3. Break pure load→use chains so other warps can issue.
4. Raise useful occupancy for memory-bound kernels.
5. Coalesce so fewer long waits per useful byte.

Related concepts