When should a region receive more tiles?
A fixed-resolution vision transformer spends the same patch count on every image of the same shape. One possible alternative is to assign finer resolution to selected regions. This lesson explores an authored complexity-driven policy. It is not an implementation of InternVL or Qwen2-VL, and it does not measure recognition quality.
The policy has four decisions: score regions, choose tile counts, resize each tile, then patchify it. Resizing is essential: dividing a fixed-resolution image without changing its pixels leaves the total patch count approximately unchanged.
The Puzzle Pieces Analogy
Start with a 4×4 region grid. Each region receives 1, 4 or 9 tiles according to its authored low, medium or high score. Every tile is resized to 224×224 pixels and split into 14×14 pixel patches, producing 256 tokens. Thus sixteen coarse regions produce 16 × 256 = 4,096 tokens, while sixteen fine regions produce 16 × 9 × 256 = 36,864 tokens.
Choose a sky, city or forest and reveal the region decisions. The displayed totals are derived from the drawn tile counts. A large coarse tile preserves less spatial resolution than several smaller tiles resized to the same dimensions; a smooth-looking region can still contain a useful small object.
Authored scene scores. Each of 16 regions supplies 1, 4 or 9 tiles; every tile is resized to 224×224 and yields 256 patches of 14×14 pixels. Total: 66 × 256 = 16,896 tokens. Coarse resizing can discard useful detail; quality is not measured.
How the illustrative policy works
1. Score visual variation
The figures use authored entropy and edge values. The local score example combines them as 0.6 × entropy + 0.4 × edge density. These scores describe variation, not guaranteed semantic importance. Noise may have high entropy; faint text may have low edge strength. No network learns the scores or weights in this explorer.
2. Select a region resolution
A threshold rule selects one, four or nine tiles. The strategy explorer uses a separate 12×12 authored region map so that threshold effects are easier to inspect. Its per-tile rule remains exactly the same: 256 tokens after resizing. Fixed means four tiles per region; maximum subdivision means nine; adaptive selects among one, four and nine.
- Total Tiles
- 1,020
- Token Count
- 261,120
- Efficiency Score
- 21.3%
Authored 12×12 region scores. Each region supplies 1, 4 or 9 tiles. Resize every tile to 224×224, then use 14×14 pixel patches: 256 tokens per tile. These counts do not measure model quality.
3. Resize and patchify
Resize each selected tile to 224×224, then extract nonoverlapping 14×14 patches. For N selected tiles:
A coarse tile compresses a larger source area into those 256 patches. This is where resolution is traded for token count. The figures omit thumbnail tokens, overlapping crops and token-merging stages. A deployed model may add any of those and will have a different count.
4. Preserve position and validate the model
Patch representations need their original region and within-region positions. Variable tiling must match the model's positional encoding, sequence shape and training distribution. It is not automatically compatible with an unchanged pretrained vision encoder.
Inspect a complexity score
Select a region to inspect the authored entropy/edge inputs, weighted score, threshold decision and resulting token count. These are illustrative inputs followed by an exact calculation, not image measurements.
The visual-complexity lesson distinguishes authored scores from a runnable pixel-based implementation and explains why both require downstream validation.
Count tokens before claiming efficiency
The next example varies how many of the sixteen regions receive nine tiles instead of one. Token savings follow directly from that choice. Dense patch-to-patch attention has T² pairs, so halving T quarters this pair count. It does not quarter all model work: projections, MLPs, scoring, resizing and data movement also contribute.
Authored 4×4 region policy: each coarse region supplies one tile and each fine region supplies nine. Resize every tile to 224×224, then use 14×14 pixel patches: 16×16 = 256 tokens per tile.
- Coarse / fine regions
- 10 / 6
- Tiles
- 10 + 9 × 6 = 64
- Adaptive tokens
- 16,384
- All-fine baseline
- 36,864
- Token reduction
- 55.6%
- Dense pair-count reduction
- 80.2%
These are analytical counts for a hypothetical preprocessing policy. Coarse tiles discard spatial resolution. Dividing an image without resizing would not reduce total patch count. Quality, latency and memory are not measured; tile-scoring overhead and model integration remain separate.
No accuracy, latency, memory, power or medical outcome is measured here. Evaluating a real policy requires a model, dataset, task metric, hardware, precision, batch size and timing boundary, including the preprocessing overhead.
Different forms of adaptive vision
The names describe different decisions. InternVL 2.5's published preprocessing chooses an image grid from aspect ratio and image area. Qwen2-VL's naive dynamic resolution varies image resolution and visual sequence length. Neither source establishes this page's entropy-threshold region rule.
Fixed grid
Choose one image resolution and patch size.
A fixed input produces a fixed token count.
Aspect-ratio tiling
Choose an image grid from aspect ratio and area.
InternVL 2.5 dynamic_preprocess; not regional entropy scoring.
Dynamic resolution
Adapt the whole-image resolution and visual token sequence.
Qwen2-VL naive dynamic resolution; not this region policy.
Illustrative region refinement
Use authored scores to choose 1, 4 or 9 resized tiles per region.
The policy on this page; quality and overhead are unmeasured.
Token selection
Keep a subset of already defined patches or representations.
A separate operation from resizing and splitting regions.
Sources: InternVL 2.5 implementation; Qwen2-VL paper.
Failure modes to test
- Missed detail: evaluate small, faint and low-texture targets, including targets in apparently simple regions.
- Boundary artifacts: inspect objects crossing tile borders and ensure positional relationships remain usable.
- Distribution shift: exercise every tile configuration at evaluation; training only on one configuration can leave other paths unsupported.
- Preprocessing overhead: measure the complete workload, including complexity scoring and tile resizing.
Related concepts
Trace how local windows, shifted cross-window exchange, and patch merging turn one high-resolution token grid into a multi-scale vision hierarchy.
Understand dilated (atrous) convolutions: how dilation rates expand receptive fields without extra parameters and how to avoid gridding artifacts.
Compare head sharing with explicit projection dimensions and KV-cache bytes. Separate exact memory ratios from measured quality and latency, and see why FlashAttention is an independent choice.
Trace how a learned CLS row joins image patches, gathers evidence through self-attention, and becomes the image-level classification readout.
Trace how grouped-query attention keeps independent query heads while sharing fewer key/value heads, projections, and compact KV-cache rows during LLM decoding.
Understand normalized kernel attention, low-rank sequence projections and their trade-offs, and compare them with sparse attention and exact tiled softmax.
