Visual representations introduced the idea that an image encoder turns pixels into useful features. A convolutional encoder can retain a grid of features. A Vision Transformer instead processes a sequence of vectors. How does a two-dimensional image become that sequence?
The first step is to cut the image into patches and turn each patch into one vector. This page follows that conversion up to the input of the transformer. The conversion sets the size and meaning of the input; later attention layers learn relationships between its entries.
Count the patches before building the vectors
An image has height , width , and channels. Divide it into square, non-overlapping patches with side length . Assume both image dimensions are divisible by and there is no padding. The grid then has rows and columns, giving
patches. Number them from left to right across the first row, then continue on the next row. That consistent ordering turns the grid into a sequence without discarding the pixels inside each patch.
For a RGB image and patches, the grid is and the sequence contains 196 patch tokens. A patch is a fixed region, not an object proposal: a face can cross several patch boundaries, and one patch can contain both foreground and background. If a dimension is not divisible by the patch side, a real pipeline must choose a policy such as resizing, cropping, or padding. The formula above assumes that choice has already been made.
Flattening changes the arrangement
Each patch contains scalar values. Flattening places these values in a fixed order in one vector. For example, a grayscale patch with rows and becomes . For RGB, each spatial position contributes three channel values, in a consistent channel order.
Flattening itself has no learned parameters and does not summarize the patch: it rearranges the same values. Its output is a sequence with shape . This raw width depends on the patch size and number of channels, so the next step maps it to the width the model expects.
A shared projection produces patch embeddings
Let be flattened patch , represented as a row vector. A learned matrix and optional bias produce its embedding:
The result has numbers. The same matrix and bias apply to every patch, so equal raw patches receive equal embeddings at this stage, wherever they occur in the image. Training adjusts these weights through the model's learning objective; there is no separate matrix for each image location. The ViT paper, Section 3.1, defines this shared trainable projection.
Change the patch side in the explorer to compare sequence lengths, then change the embedding width. Selecting different patches shows how a shared rule can produce different vectors from different input values.
Authored grayscale pixels, rows: [0, 1, 2, 3]; [1, 2, 3, 0]; [2, 3, 0, 1]; [3, 0, 1, 2]. Select a patch in reading order.
Patch P1 flattens to [0, 1, 1, 2].
Each input value uses weight 1 for output 1, −1 for output 2, and 0.5 for output 3 when present. Bias is zero.
P1 → [4, -4]
Authored image and weights; outputs are exact arithmetic, not measured pretrained features. Every patch uses the same projection. No position vectors or attention are applied.
The explorer deliberately uses simple fixed weights so every output can be checked by hand. A trained projection learns many different weighted combinations of the pixels. The width is an architectural choice; it need not equal , and the conversion need not be a compression. Even equal input and output widths do not guarantee an invertible mapping.
In the official Google implementation, a convolution with kernel size and stride performs patch extraction and projection together, followed by a reshape into a sequence. Its filters are shared over positions, just as the matrix above is shared across patches. See the embedding layer in models_vit.py. This is an implementation of the patch projection, rather than a whole convolutional feature hierarchy.
Sequence length and vector width are different choices
The projected sequence has shape : says how many locations enter the model, while says how many numbers describe each location. For a batch of images, the shape is .
Keeping a input and changing the patch side from 16 to 32 reduces the grid from to , so falls from 196 to 49. The model can keep the same . Larger patches place more local pixels into each token and create fewer tokens; smaller patches retain a finer input grid. Neither setting alone establishes better accuracy.
With fixed , and , increasing image resolution increases the token count but does not change the projection's parameter count: weights, plus values if a bias is used. More tokens mean more downstream work, including more pairwise interactions in ordinary global self-attention. That arithmetic is not a measured latency prediction.
Patch embeddings are the beginning of the encoder
At this point, each embedding depends only on the pixels in its own patch. There has been no exchange of information between patches, no object recognition, and no image-level prediction. The initial vectors become contextual representations only as later encoder blocks process them together.
There is also a missing ingredient: where did each patch come from? A sequence is stored in an order, but a shared content projection does not itself encode that order. ViT adds position embeddings to identify sequence locations before its attention blocks. For classification, the original ViT also prepends a learned class token, so 196 patch tokens become 197 input tokens. That extra token is not another image patch. The paper's model overview and input equation show these as separate steps.
Continue with positional embeddings to see how location enters the vectors, then the ViT walkthrough to follow them through attention and an image-level readout. To follow the learning branch, self-supervised learning and masked prediction asks what a model can learn from these image regions when human labels are unavailable.
Related concepts
Compare anchor-based vs anchor-free object detection: Faster R-CNN and RetinaNet anchors vs FCOS and CenterNet point-based methods.
Learn how DETR assigns unordered object targets to prediction slots, why greedy choices can fail, and how matching differs from training loss and evaluation.
Learn how normalized box coordinates, L1 loss, IoU, and Generalized IoU guide object localization, including disjoint boxes and containment limits.
Convert between xyxy, xywh and center-size boxes, normalize coordinates correctly, and avoid mistakes when resizing, clipping or flipping detection labels.
Follow a 3D world point through camera extrinsics, depth division and pixel intrinsics, then see why one image point cannot determine its depth.
Follow a query from a spatial reference through learned offsets, bilinear feature sampling and weighted aggregation, with exact non-square coordinate arithmetic.
