An image coordinate tells us where a sample sits in an array. Camera geometry asks how a location in the world became that image coordinate. A person farther away occupies fewer pixels, and moving the camera changes a stationary object's image position. Both effects follow from expressing a point relative to the camera, then dividing by its depth.
The ideal pinhole camera makes this relationship precise. It maps one 3D point to one 2D location; it does not describe color, blur or which surface hides another. The useful question is: given a world point and a camera, where would that point project?
First agree on the axes
Write a world point as . The world frame is a fixed reference for the scene. A separate camera frame moves with the camera: its origin is the optical center, X points right, Y points down, and Z points forward. These directions follow the camera's view, rather than fixed compass directions.
In the image, increases rightward and increases downward. Image coordinate (u, v) still corresponds to array indexing image[v, u]. The camera coordinates are distances in scene units; the image coordinates are measured in pixels. Dividing one by the other without accounting for that difference is a common source of errors.
Extrinsics put the world in the camera frame
Let be the camera center expressed in world coordinates. Subtracting it gives the displacement from the camera to the point. A rotation then expresses that displacement along the camera's axes:
Here is the world-to-camera rotation. Its transpose maps camera directions back to the world. The extrinsic translation is not generally the camera's world position : it includes both the opposite sign and the rotation.
For an unrotated camera at , a world point becomes . Moving the camera right reduces the point's camera-relative X coordinate. The world point has not moved.
This convention matches the world-to-camera transformation in OpenCV's camera model. Other graphics systems may use different axis directions or store the inverse camera pose. Establish the mapping before copying a matrix into a vision pipeline.
Depth division creates perspective
For a point in front of this camera, . Similar triangles give the coordinates on a virtual image plane one unit in front of its center:
These normalized camera coordinates are dimensionless. They are not pixel coordinates, and they are not the same as scaling an image tensor into the range 0–1. Doubling depth while keeping X and Y fixed halves the point's distance from the optical axis in this normalized plane.
Placing the virtual plane in front keeps its orientation aligned with the image. A physical sensor lies behind the pinhole and receives an inverted image; the front-plane convention avoids carrying that inversion through the equations.
Intrinsics turn a direction into pixels
Depth division establishes a direction. The intrinsic parameters convert it to the sensor's pixel grid:
The focal lengths and are measured in pixels, rather than millimeters. The principal point is where the optical axis intersects the image plane; it is often near the image center but need not equal it. The explorer fixes it at (320, 240) and sets both focal lengths to the same value. The zero-skew intrinsic matrix is:
With , and principal point (320, 240), the normalized position is (0.25, 0.25) and the projected pixel is (420, 340). Doubling the focal lengths changes it to (520, 440): the offset from the principal point doubles. The 3D point and its depth stay fixed.
For a fixed sensor resolution, a larger focal length gives a narrower field of view. Resizing an image also changes its pixel calibration: scaling both dimensions by a factor scales the focal lengths and principal point by that factor. Cropping shifts the principal point by the crop's origin. These are changes in the pixel coordinate system, not changes in the scene. OpenCV documents intrinsic parameters and image scaling.
The explorer uses exact calculations on an authored point, with world Y fixed at 0.5 scene units. Start by moving the camera right, then change the focal length. Choose Outside and Behind to compare a point beyond the image edge with one that has negative camera depth. Same ray adds a second point twice as far from the camera, including when the camera is translated or rotated.
World top view: +Z right, +X down
Square: camera center C. Arrow: camera forward axis. Teal: P. World grid spacing: 2 scene units.
Image: u right, v down · 640 × 480 pixels
Cross: principal point (320, 240). Teal dot: projected point.
Exact calculation of an authored scene. From Start, move the camera right: the fixed world point moves left in the image. Increase focal length: its offset from the principal point grows.
A projected location is not a visibility test
Three cases need separate treatment:
- Positive camera depth and inside image bounds: a candidate image location exists. A nearer surface may still occlude the point.
- Positive camera depth and outside image bounds: the point projects beyond the recorded image. It remains in front of the camera.
- Zero or negative camera depth: depth division is undefined at zero, and a negative-depth point is behind the forward-looking camera. Its algebraic ratio is not an observation in this camera's image.
The explorer checks the continuous coordinate bounds and , without rounding to an array element. It does not render surfaces or test occlusion. Near-zero camera depths are also excluded from projection to avoid unstable division.
One image point leaves depth undetermined
Suppose the calibration is known and a pixel has been observed. Reversing the intrinsic conversion gives a camera direction, but not a distance:
Every positive choice of produces a point on the same ray. In the world frame that ray is . The explorer's amber point scales the displacement from the camera center by two, so both points project to the same pixel even after changing the camera pose.
A single calibrated image point therefore cannot determine a unique 3D position. A second view, a known object size, a depth sensor, or additional scene assumptions can supply information that the projection discarded. A learned depth estimate also relies on information beyond this one point's coordinates.
Where the ideal model stops
Real lenses can bend straight lines through distortion. In a calibrated distorted-camera model, normalized coordinates are distorted before focal scaling and principal-point offsets are applied. The explorer uses zero distortion, zero skew and a fixed principal point; it is an illustration of projection, not a camera calibration or reconstruction result. OpenCV describes radial and tangential distortion alongside the ideal model.
Projection explains how camera motion and scene depth can change image positions. Optical flow next asks which image location corresponds to a point in another frame. Flow can be estimated directly in 2D; knowing the camera geometry helps interpret it, but a flow vector alone does not uniquely reveal 3D motion or depth.
Related concepts
Compare anchor-based vs anchor-free object detection: Faster R-CNN and RetinaNet anchors vs FCOS and CenterNet point-based methods.
Learn how DETR assigns unordered object targets to prediction slots, why greedy choices can fail, and how matching differs from training loss and evaluation.
Learn how normalized box coordinates, L1 loss, IoU, and Generalized IoU guide object localization, including disjoint boxes and containment limits.
Convert between xyxy, xywh and center-size boxes, normalize coordinates correctly, and avoid mistakes when resizing, clipping or flipping detection labels.
Follow a query from a spatial reference through learned offsets, bilinear feature sampling and weighted aggregation, with exact non-square coordinate arithmetic.
Follow coarse feature maps into aligned pixel predictions: nearest and bilinear interpolation, transposed convolution, output shapes, skip fusion, and resizing logits before selecting labels.
