SoA vs AoS: Data Layout Optimization

You asked for eight x values. The bus brought six other fields.

Particles, vertices, rows, embeddings — anything with several fields per item has a layout choice. Put each object’s fields together (Array of Structures), or put each field’s values together (Structure of Arrays). The CPU and GPU never load “just a float.” They load a cache line or a memory segment. Whatever else sits on that line rides for free — or wastes the trip.

This page is that trip: what the ribbon looks like, what a line actually contains, why SIMD and warps care, and when a hybrid tile is the sane middle.

Two numbers

1. 64 bytes
A typical CPU cache line. Four-byte floats → 16 floats per free ride. Layout decides how many of those 16 you actually use.

2. 32 threads
A GPU warp. One instruction, 32 addresses. Contiguous SoA → one transaction class. AoS stride → many segments and a bandwidth bill.

Same data, different ribbon

Eight particles, four floats each (x, y, z, m). Flip layout. Only order changes — and order is what hardware sees.

AoS matches how we talk about objects. SoA matches how batch kernels walk memory. Neither is “more correct”; each is a bet on the hot access pattern.

Useful percent of every line

You request x across the set. Hardware still fills whole lines. Teal is what you asked for; amber is what you paid for and ignored.

SIMD wants neighbors

An 8-wide vector of x is one instruction only if the eight floats already sit together. Otherwise the load becomes gather — or eight scalar loads with a smile.

Compilers auto-vectorize loops that walk SoA cleanly. They struggle when the source addresses hop by particle stride. Layout is a precondition for the vector unit, not a free lunch after the fact.

Warps want the same thing, wider

Thirty-two GPU threads reading x[tid] is the same story with higher stakes. Coalescing is layout with a warp-shaped ruler.

This is why CUDA particle codes, image planar formats, and training tensors live as SoA-ish buffers. The SM coalescing film is the same mechanism from the SM’s point of view; here the lever is the struct, not the index arithmetic alone.

Hybrid when both paths matter

AoSoA: blocks of K particles, SoA inside each block. Batch field scans stay dense inside the tile; a whole object does not scatter across the entire array.

Pick K near SIMD width (8 / 16) or a warp subgroup. Game entity systems and some HPC packs use this shape so the integrator and the rare “inspect entity 17” path both survive.

Pick from the hot path

Flip the workload. Bars are toy useful-bandwidth, not wall-clock — but the ranking matches what profilers reward when you are memory-bound.

Practical defaults

• SoA — particle / sim loops, columnar analytics, GPU kernels, any “one field, many ids.”
• AoS — trees, editors, APIs that hand out whole records, small sets that stay in L1 anyway.
• AoSoA — both a vectorized hot loop and occasional whole-object access in the same working set.
• Measure — if you are compute-bound, layout reshuffles are theater. Use cache-miss and DRAM counters first.

Habits that keep the win

1. Align hot SoA arrays (32 / 64 B) and size them to multiples of SIMD / warp width when it is free.
2. Do not half-convert a codebase — convert at system boundaries; keep one layout on the hot path.
3. Watch false sharing when threads write adjacent SoA slots; pad or privatize accumulators.
4. Profile before and after (perf stat -e cache-misses, VTune, Nsight memory).

Where it shows up

Related concepts