TL;DR
- SigLIP 2 combines sigmoid image-text alignment with decoder supervision and teacher-student objectives to train global and spatial representations.
- Training-only branches can improve an encoder without becoming released inference heads. Patch features alone do not supply boxes or segmentation outputs.
- Schedule, multilingual data and tokenizer choices are part of the recipe. NaFlex adapts image geometry within a patch budget and masks padding during attention.
- The runnable calculations, fixed-resolution recorded matching and published benchmarks provide separate evidence. The saved matching examples do not establish NaFlex, localization or dense-task performance.
What must survive in an image representation?
The authored scene below shows a tree beside a red ball on the grass. Matching it to “a tree beside a red ball” asks for a useful summary. Matching “the red ball” to a region asks the representation to preserve which features belong where. A single pooled vector hides that distinction from the reader: it can produce a plausible match without showing whether the underlying patch features support a spatial task.
Start with the paths below. Follow the image to its pooled embedding, then follow its patch features to the extra training objectives. The scene, captions, boxes and feature values in these mechanics figures are authored teaching examples. Their arithmetic is inspectable; they are not predictions from a checkpoint.
The 2025 preprint presents SigLIP 2 as a combined training recipe. This review is dated October 18, 2026. The useful distinction throughout is between what supervises the encoder during training and what a released encoder returns at inference. A training branch can change the representation even when that branch is absent from the released model.
That makes the first question concrete: how can language supervision reach the patch features before they are pooled?
Give language supervision a spatial route
Choose one decoder task below. With a phrase as input, the target can be a box. With a box as input, the target can be a phrase. With neither region prompt, the target can describe the image. Reversing the direction changes what the decoder must recover from the same visual features.
Section 2.2 places the LocCa decoder over unpooled features and combines its loss equally with sigmoid alignment. All three tasks supervise each example. Caption prediction uses the parallel masked-token mode half the time. The decoder is excluded from the model release.
For an autoregressive teaching example, a target sequence has the familiar factorization below. Here H denotes visual patch features, c task conditioning, and y_t a target token. This factorization describes the causal mode only; parallel caption prediction uses masked contexts instead of the target prefix.
The key consequence is a gradient path. To predict “red ball” from a region prompt, the decoder must access information useful for that region through its cross-attention to the visual sequence. The prompt and target direction tell us what supervision is available. They do not establish that an arbitrary released checkpoint can generate a box on demand.
This also explains why a colorful patch map would be insufficient evidence. A box prediction requires a specified output head and evaluation protocol. A patch tensor is an intermediate representation from which such a head might learn.
Decoder supervision supplies language-linked targets, but language does not describe every visual detail. The next branch adds a second kind of target: another view of the image itself.
Let the whole image teach its parts
Select a local crop, then switch to masked patches. In the first case, the student has incomplete spatial context. In the second, the grid remains aligned but some student patch embeddings are hidden. Keeping those cases separate prevents an easy mistake: treating every crop as if its patch indices directly matched the full image.
Section 2.3 uses one global teacher view and eight local student views. Masked prediction uses the same global view on both sides, hiding half the student patches and targeting teacher features at those locations. These branches begin at 80% of training.
Two operations matter independently. A stopped target blocks the current backward pass. An exponential moving average changes the teacher’s parameters over time:
If the teacher is [2, 4], the student is [6, 8], and an illustrative momentum is 0.75, the updated teacher is [3, 5]. That calculation says nothing about the appropriate production momentum schedule. It simply shows that the teacher can move without receiving gradients through the target.
The executable reference below uses squared feature distance to make these paths visible. That distance is a teaching surrogate. SigLIP 2 inherits a consistency objective from SILC/TIPS; the toy does not reproduce its projection heads, distributional targets, temperatures or centering. A small distance to a detached vector is useful for testing correspondence, but it is not sufficient to show that an encoder learns noncollapsed, useful representations.
Those extra objectives have a cost and a time of introduction. Understanding the recipe therefore requires a schedule, not just a list of loss names.
Treat schedule and data policy as part of the model
Scrub the training timeline before opening the data-policy view. The display separates when an objective is active from how examples are selected. Switching a checkbox in an illustration would not be a controlled ablation: changing the training path also changes compute, optimization history and the representations available to later stages.
The NaFlex branch starts at 90% and omits the added self-supervised objectives during adaptation. Small-model ACID training is an additional sample-selection stage. The selector in this teaching view does not implement ACID’s scoring algorithm.
Multilingual behavior also belongs to the recipe rather than the drawing of the network. The training data mixture is 90% English and 10% non-English, with the multilingual Gemma tokenizer. A tokenizer is part of the model interface: retaining the old tokenizer while swapping weights is not a faithful checkpoint substitution.
For interpreting the schedule, ask three separate questions. Which inputs reach a branch? Which targets define its learning signal? Which parameters are updated? That checklist distinguishes a training-only decoder, a stopped teacher, and a data selector without confusing any of them with an inference head.
The paper’s overview brings those paths back together. Its percentages indicate portions of training, not loss weights; the diagram summarizes the main training recipe rather than every adaptation branch.
Original paper, Figure 1. Michael Tschannen and colleagues, 2025. Rasterized from the original vector figure; diagram content unchanged. Source and CC BY 4.0 license.
The remaining design choice is visible before any objective is evaluated: how does a rectangular image become a sequence of patches?
Preserve shape within a patch budget
Try a panorama and then a portrait. Square resizing has a fixed sequence length but changes proportions. A rectangular patch grid can preserve the shape more closely, while a budget limits how many visual tokens the model processes. Padding fills unused sequence positions; it is not extra image content.
For an already chosen grid with R rows, C columns and budget K, the accounting is simple:
For example, a 2 × 3 grid under a budget of eight has six real patch coordinates and two invalid positions. This identity checks bookkeeping, not the resize algorithm. Choosing the grid, resampling the image and resizing positional embeddings are additional operations. The figure labels the scope of its geometry model; the CPU helper deliberately accepts a chosen grid instead of pretending to reproduce official preprocessing.
The paper’s NaFlex description masks padding in attention, including pooling. Ignoring invalid positions only after attention would be too late: those positions could already have changed the real tokens. Likewise, a sequence of the correct length is not enough to establish correct geometry if patch coordinates or positional embeddings are inconsistent.
A larger token budget also changes computational cost. More patches may preserve useful small details, but a sequence budget is not an accuracy guarantee. That is why a geometry illustration and a task-conditioned measurement answer different questions. Before inspecting measurements, we can make the arithmetic and gradient contracts executable.
From the paper to a runnable calculation
The inherited sigmoid objective tests every image against every caption in a paired batch. Signed labels are positive on the matching diagonal and negative elsewhere. The original SigLIP derivation explains the loss in detail; the important reduction here is the sum over all pairs divided by the number of examples, not by the number of pairs.
The inherited implementation convention is pinned to big_vision revision 642c086e68b5. The rows distinguish exact inherited arithmetic, generic likelihood, and deliberately narrowed teaching approximations. Code is extracted from the executable file, so the displayed implementation and download can be checked byte for byte.
These rows connect source claims to executable, offline CPU mechanics. Every excerpt is extracted from the downloadable Python file. Tildes identify authored surrogate losses, not the paper’s training objective.
1. Preserve the inherited pairwise reduction
Original paper
SigLIP 2 retains the original sigmoid alignment objective.
Math and tensor shapes
Exact inherited reduction for a paired batch: sum N² pair terms and divide by N. The vectors are normalized with norm plus 1e-8, following the pinned big_vision convention linked in the article. A mean over all matrix cells would shrink the objective by N. Learned log scale and bias remain on the gradient path.
image, text [N, D] → logits [N, N] → scalar
Python implementation
def alignment_loss(image, text, log_scale, bias):
"""Compute inherited all-pairs sigmoid sum/N, not mean over N squared."""
_same_features(image, text, 2)
for parameter in (log_scale, bias):
if (not isinstance(parameter, torch.Tensor) or parameter.ndim != 0
or parameter.dtype != image.dtype or parameter.device != image.device
or not torch.isfinite(parameter)):
raise ValueError("scale and bias must be matching finite scalars")
scale = log_scale.exp()
if not torch.isfinite(scale) or scale <= 0:
raise ValueError("exponentiated scale must be finite and positive")
image = image / (image.norm(dim=-1, keepdim=True) + 1e-8)
text = text / (text.norm(dim=-1, keepdim=True) + 1e-8)
logits = scale * (image @ text.T) + bias
n = image.shape[0]
labels = 2 * torch.eye(n, dtype=image.dtype) - 1
return F.softplus(-labels * logits).sum() / n2. Inspect token likelihood without inventing a decoder
Original paper
The target sequence depends on the task and its conditioning.
Math and tensor shapes
Generic token negative log likelihood, averaged over valid target positions V. This is not an implementation of LocCa serialization, its task weighting, or its attention masks. It accepts already aligned logits and target IDs; it does not shift targets automatically. Autoregressive contexts and parallel masked contexts must be constructed upstream. The fixture verifies that ignored padding contributes no gradient.
logits [B, T, vocabulary], targets [B, T] → scalar
Python implementation
def decoder_token_nll(logits, targets):
"""Generic mean token NLL; inputs/attention/task serialization are external."""
_features(logits, 3) # [batch, target positions, vocabulary]
if (not isinstance(targets, torch.Tensor)
or targets.shape != logits.shape[:2] or targets.dtype != torch.int64
or targets.device.type != "cpu"):
raise ValueError("targets must be matching CPU integer token IDs")
valid = targets != -100
if not valid.any() or ((targets[valid] < 0)
| (targets[valid] >= logits.shape[-1])).any():
raise ValueError("need valid vocabulary IDs; -100 denotes padding")
return F.cross_entropy(logits[valid], targets[valid])3. Separate teacher updates from backpropagation
Original paper
EMA updates parameters; stopping a target controls gradients.
Math and tensor shapes
Exact EMA arithmetic on a tiny parameter vector. At m=0.75, teacher [2,4] and student [6,8] produce [3,5]. The helper returns a fresh vector under no_grad, leaving both inputs unchanged. A real trainer copies the result into its teacher after updating the student and supplies its actual momentum schedule. No momentum schedule is inferred here.
teacher [D], student [D], scalar m → detached [D]
Python implementation
@torch.no_grad()
def ema_update(teacher, student, momentum):
"""Return EMA parameters without a gradient path; leave inputs unchanged."""
_same_features(student, teacher, 1)
if not math.isfinite(momentum) or not 0 <= momentum <= 1:
raise ValueError("momentum must be in [0, 1]")
return momentum * teacher + (1 - momentum) * student4. Make a stopped global target visible
Original paper
The paper uses a learned projection space and inherited consistency loss.
Math and tensor shapes
Teaching surrogate only: squared feature distance exposes correspondence and stop-gradient. It is not the paper’s SILC/TIPS consistency objective, and omits distributional targets, temperatures, centering and projection heads. One teacher vector broadcasts across this image’s local views. A test checks its exact value and confirms that only the student receives gradients. This loss alone does not prevent representation collapse.
local [views, D], global [1, D] → scalar surrogate
Python implementation
def global_consistency(student_local, teacher_global):
"""TEACHING SURROGATE: squared feature error, not official consistency loss."""
_features(student_local, 2) # [number of local views, channels]
_features(teacher_global, 2) # [1, channels], one image in this toy
if (teacher_global.shape != (1, student_local.shape[1])
or teacher_global.dtype != student_local.dtype):
raise ValueError("one global teacher vector must match channel width")
target = teacher_global.detach().expand_as(student_local)
return (student_local - target).square().mean()5. Select hidden, valid positions before reducing
Original paper
Masked prediction targets feature vectors rather than RGB pixels.
Math and tensor shapes
Teaching surrogate only, with the same caveat as the preceding row. M marks hidden positions; A marks valid positions. The extra validity mask is defensive bookkeeping, not a claim that the NaFlex adaptation uses this objective. Empty selection raises an error. Tests ensure visible positions and padding have zero gradient and teacher targets remain stopped.
features [B, P, D], masks [B, P] → scalar surrogate
Python implementation
def masked_consistency(student, teacher, masked, valid):
"""TEACHING SURROGATE: error only at valid masked feature positions."""
_same_features(student, teacher, 3) # [batch, patches, channels]
for mask in (masked, valid):
if (not isinstance(mask, torch.Tensor) or mask.dtype != torch.bool
or mask.shape != student.shape[:2] or mask.device.type != "cpu"):
raise ValueError("masks must be matching CPU boolean [B, P] tensors")
selected = masked & valid
if not selected.any():
raise ValueError("need at least one valid masked position")
difference = student[selected] - teacher.detach()[selected]
return difference.square().mean()6. Verify a chosen patch grid and its padding
Original paper
NaFlex ignores padded tokens in attention, including attention pooling.
Math and tensor shapes
This helper takes an already chosen integer grid. A 2×3 grid with budget 8 has six coordinates and two invalid padding entries. It does not implement the official aspect-preserving resize search, image interpolation, positional embedding resize or attention masking itself. Coordinate entries are for valid patches only; a caller must pad its coordinate tensor consistently if needed.
rows R, columns C, budget K → P coordinates and K mask values
Python implementation
def grid_layout(rows, columns, budget):
"""Bookkeep a chosen grid; does NOT choose NaFlex resize/interpolation."""
for value in (rows, columns, budget):
if isinstance(value, bool) or not isinstance(value, int) or value < 1:
raise ValueError("grid dimensions and budget must be positive integers")
used = rows * columns
if used > budget:
raise ValueError("chosen grid exceeds token budget")
return {
"coordinates": [[row, col] for row in range(rows) for col in range(columns)],
"valid": [True] * used + [False] * (budget - used),
"used": used,
"padding": budget - used,
}Smoke performs zero optimizer steps. It reports actual toy losses, gradient norms, stopped teacher targets, EMA arithmetic, seed, software version and source SHA-256. Its sum of independent test losses is not the paper’s weighted total objective.
siglip2.pyrequirements-siglip2.txt
python -m pip install -r requirements-siglip2.txt python siglip2.py --smoke --seed 7
A passing toy test establishes a local contract. It can show that the teacher receives no gradient, that only selected patch positions contribute, or that the sigmoid denominator is correct. It cannot establish useful learned features or reproduce the reported benchmarks. To inspect learned behavior, the next section uses a separate recorded checkpoint export.
Recorded matching with a released checkpoint
The explorer uses saved outputs from google/siglip2-base-patch16-224, revision 75de2d55ec2d0b4efc50b3e9ad70dba96a7b2fa2. Its artifact records CPU float32 inference, direct bilinear resizing to 224 × 224, lowercased captions and a 64-token limit. The export contains three licensed photographs, each compared with nine authored captions. Reconstructing logits from its saved embeddings, learned scale and bias differs from the recorded logits by at most 4.77 × 10⁻⁶. That check validates score arithmetic, not semantic correctness. Inspect the input pixels and complete candidate set together; a prompt is part of the experiment.
These are recorded pretrained outputs. The controls select saved image-caption comparisons; they do not run a model in the browser. Independent sigmoid scores need not sum to one. They should not be read as calibrated probabilities that a caption is true. Two similar captions can both score highly, and a candidate omitted from the set has not been evaluated by this display.
The artifact’s pinned model source and explorer provenance identify what was run. Fixed-resolution matching supplies no NaFlex, localization or segmentation evidence, and this small set does not measure improvement over original SigLIP. That comparison belongs to a defined benchmark protocol.
Read results with their evaluation conditions
Choose an evidence category below. Keep the model size, input resolution, metric direction and evaluation protocol attached to each value. A retrieval score, a dense probe and a shape-sensitive evaluation cannot be added into a meaningful overall score simply because all are reported as numbers.
The paper’s evaluation section supplies the measurements shown here. These are published paper measurements, separate from the local matching export and authored mechanics. A dense-task result includes the downstream head and training protocol; it is not the output of passing an image through an embedding model alone.
For adoption, first match the evidence to the intended use. A retrieval system needs relevant ranking measurements and its own candidate distribution. A document workflow needs shape and text-detail evaluation. A dense prediction pipeline needs a suitable downstream head. Strong performance in one setting motivates testing in another, but does not remove that testing step.
What to take into an implementation
SigLIP 2 is useful to think about as a representation trained under several complementary demands. Pooling supports a global match. Decoder supervision puts language-linked demands on spatial features. Teacher targets make partial or masked views answer to a more complete view. Geometry determines which visual information reaches those objectives in the first place.
The limits of this review are concrete. The CPU file checks arithmetic on synthetic tensors and does not recreate WebLI, automatic annotations, ACID, full LocCa, SILC/TIPS or NaFlex preprocessing. The saved inference is a small fixed-resolution matching demonstration. Published results remain conditioned on their reported models and evaluation setups. Keeping these boundaries visible lets each piece of evidence do the job it can support.
Related reading
- Original SigLIP: follow the pairwise objective, learned scale and bias before extending the training recipe.
- CLIP: compare batch-normalized contrastive matching with independent pair scores.
- Cross-attention: inspect how a decoder accesses a visual sequence.
- SigLIP 2 paper, version 1: training details, downstream protocols and complete tables.
- Official checkpoint release: released model families and inference entry points.
