SoA vs AoS: Data Layout Optimization

Why Data Layout Matters

When storing collections of multi-field data—particles, vertices, database records—the memory layout choice between Array of Structures (AoS) and Structure of Arrays (SoA) can result in 10-100x performance differences. This single architectural decision affects CPU cache efficiency, SIMD vectorization, and GPU memory coalescing.

The Library Analogy

Imagine organizing a library of books, where each book has: title, author, year, and genre.

AoS (Traditional Shelving): Each book sits together with all its information on one shelf card.

• To find all titles? You must visit every single shelf and read each card.
• Great when you need everything about one specific book.

SoA (Columnar Organization): All titles on one shelf, all authors on another, all years on a third.

• To find all titles? Just visit the titles shelf—done!
• Perfect when you only need one piece of information from every book.

This is exactly how CPUs access memory. SoA lets the CPU grab what it needs without wading through irrelevant data.

Understanding the Two Layouts

Array of Structures (AoS)

Groups all fields of each object together in memory. Each particle's x, y, z, velocity, and mass are stored contiguously. Natural for object-oriented thinking.

Structure of Arrays (SoA)

Groups each field into separate contiguous arrays. All x-values together, all y-values together. Optimal for batch processing and SIMD operations.

Why Layout Matters: The Cache Efficiency Story

1. CPU requests a single value — You ask for particle[0].x—just 4 bytes of data.

2. Hardware loads entire cache line — The CPU doesn't fetch 4 bytes. It loads a full 64-byte cache line containing that address.

3. Layout determines what comes along — With AoS, you get x, y, z, vx, vy, vz, mass, charge for ONE particle (useful if you need all fields). With SoA, you get x₀, x₁, x₂... x₁₅ for 16 particles (useful if processing all x-values).

4. Unused data wastes bandwidth — If you only need x-values across all particles, AoS wastes 87.5% of loaded data. SoA uses 100%.

Cache Efficiency Comparison

Access Pattern	AoS Efficiency	SoA Efficiency
All fields of one object	100%	12.5%
Position only (x, y, z)	37.5%	100%
Single field (x) across all	12.5%	100%

The pattern is clear: AoS wins for random access to complete objects. SoA wins overwhelmingly for batch operations on specific fields—which is the common case in simulations, games, and data processing.

Interactive Memory Explorer

Explore how different data layouts affect memory access patterns and cache efficiency:

SIMD Vectorization

Modern processors don't just process one value at a time. SIMD (Single Instruction, Multiple Data) processes 4-16 values simultaneously using vector instructions like AVX2.

The Problem with AoS: To process 8 x-values, the CPU must gather them from 8 different memory locations (scattered across 8 particle structures). This "gather" operation is slow.

The SoA Advantage: All 8 x-values are already adjacent in memory. One instruction loads all 8. One instruction processes all 8. One instruction stores all 8.

GPU Memory Coalescing

GPUs are even more sensitive to memory layout. A warp of 32 threads accessing data:

• AoS: 32 threads need 32 separate memory transactions—the GPU serializes these, destroying parallelism.
• SoA: 32 threads access 32 adjacent floats—hardware coalesces into 1-2 transactions.

Performance Comparison

When to Use Each Layout

Choose AoS When:

• Object-oriented design is paramount
• Random access to complete objects dominates
• Small working sets fit in cache
• Using pointer-based structures (linked lists, trees)

Choose SoA When:

• Batch processing many objects
• SIMD optimization is critical
• GPU computing (CUDA/OpenCL)
• Scientific simulations with large datasets

Consider AoSoA Hybrid:

Group objects into SIMD-width blocks (8 for AVX2, 16 for AVX-512). Each block uses SoA internally. This provides cache locality of AoS with vectorization benefits of SoA.

Common Pitfalls to Avoid

1. Premature Optimization

Converting to SoA without measuring first. Memory layout only matters when memory bandwidth is the bottleneck.

Solution: Profile with cache miss counters before restructuring. If computation-bound, layout won't help.

2. Forgetting Alignment

SIMD instructions require data aligned to 16/32/64-byte boundaries. Unaligned access causes crashes or severe slowdowns.

Solution: Use alignas(32) or aligned allocators. Ensure array sizes are multiples of SIMD width.

3. False Sharing in Multi-threaded Code

Different threads writing to arrays that share cache lines causes constant invalidation.

Solution: Pad arrays to cache line boundaries (64 bytes). Use thread-local accumulators.

4. Mixing Layouts Inconsistently

Half the codebase uses AoS, half uses SoA. Constant conversion overhead negates benefits.

Solution: Choose one layout for your hot path and stick with it. Convert at system boundaries only.

Real-World Applications

Domain	Application	Why SoA/AoS
Game Engines	Unity DOTS, Unreal Mass	SoA enables millions of entities at 60fps
Scientific Computing	LAMMPS, GROMACS molecular dynamics	SoA with SIMD achieves 10x+ speedups
Columnar Databases	Apache Parquet, Arrow, DuckDB	SoA (columnar) for efficient analytical queries
Machine Learning	PyTorch, NumPy tensors	SoA for optimal GPU batch processing
Image Processing	FFmpeg planar formats	SoA (planar RGB) enables SIMD color processing
Financial Systems	HFT price feeds	SoA for rapid scanning across instruments

Key Takeaways

1. Layout is a 10-100x decision — Not a micro-optimization, this is architectural.

2. SoA wins for batch processing — If you touch one field across many objects, SoA is almost always faster.

3. AoS wins for random access — If you need all fields of random objects, AoS avoids pointer chasing.

4. SIMD and GPUs demand SoA — Modern hardware parallelism requires contiguous data to achieve peak performance.

5. Measure first — Profile cache misses before restructuring. The "wrong" layout for your access pattern costs 8-10x.

Profiling Tools

Use these tools to measure the impact of layout changes:

• Intel VTune — CPU cache analysis and memory bandwidth
• NVIDIA Nsight — GPU coalescing metrics
• Linux perf — perf stat -e cache-misses for quick cache analysis

Related Concepts

• CPU Cache Lines — Understanding why layout affects cache efficiency
• Memory Hierarchy — GPU memory coalescing patterns
• CPU Optimization — Broader optimization strategies