Skip to main content

Camera Geometry and Projection

Follow a 3D world point through camera extrinsics, depth division and pixel intrinsics, then see why one image point cannot determine its depth.

An image coordinate tells us where a sample sits in an array. Camera geometry asks how a location in the world became that image coordinate. A person farther away occupies fewer pixels, and moving the camera changes a stationary object's image position. Both effects follow from expressing a point relative to the camera, then dividing by its depth.

The ideal pinhole camera makes this relationship precise. It maps one 3D point to one 2D location; it does not describe color, blur or which surface hides another. The useful question is: given a world point and a camera, where would that point project?

First agree on the axes

Write a world point as Pw=(Xw,Yw,Zw)TP_w = (X_w, Y_w, Z_w)^T. The world frame is a fixed reference for the scene. A separate camera frame moves with the camera: its origin is the optical center, X points right, Y points down, and Z points forward. These directions follow the camera's view, rather than fixed compass directions.

In the image, uu increases rightward and vv increases downward. Image coordinate (u, v) still corresponds to array indexing image[v, u]. The camera coordinates are distances in scene units; the image coordinates are measured in pixels. Dividing one by the other without accounting for that difference is a common source of errors.

Extrinsics put the world in the camera frame

Let CC be the camera center expressed in world coordinates. Subtracting it gives the displacement from the camera to the point. A rotation RR then expresses that displacement along the camera's axes:

Pc=R(Pw−C)=RPw+t,t=−RC.P_c = R(P_w - C) = RP_w + t, \qquad t = -RC.

Here RR is the world-to-camera rotation. Its transpose maps camera directions back to the world. The extrinsic translation tt is not generally the camera's world position CC: it includes both the opposite sign and the rotation.

For an unrotated camera at C=(1,0,0)TC=(1,0,0)^T, a world point Pw=(2,1,4)TP_w=(2,1,4)^T becomes Pc=(1,1,4)TP_c=(1,1,4)^T. Moving the camera right reduces the point's camera-relative X coordinate. The world point has not moved.

This convention matches the world-to-camera transformation in OpenCV's camera model. Other graphics systems may use different axis directions or store the inverse camera pose. Establish the mapping before copying a matrix into a vision pipeline.

Depth division creates perspective

For a point in front of this camera, Zc>0Z_c > 0. Similar triangles give the coordinates on a virtual image plane one unit in front of its center:

xn=XcZc,yn=YcZc.x_n = \frac{X_c}{Z_c}, \qquad y_n = \frac{Y_c}{Z_c}.

These normalized camera coordinates are dimensionless. They are not pixel coordinates, and they are not the same as scaling an image tensor into the range 0–1. Doubling depth while keeping X and Y fixed halves the point's distance from the optical axis in this normalized plane.

Placing the virtual plane in front keeps its orientation aligned with the image. A physical sensor lies behind the pinhole and receives an inverted image; the front-plane convention avoids carrying that inversion through the equations.

Intrinsics turn a direction into pixels

Depth division establishes a direction. The intrinsic parameters convert it to the sensor's pixel grid:

u=fxxn+cx,v=fyyn+cy.u = f_x x_n + c_x, \qquad v = f_y y_n + c_y.

The focal lengths fxf_x and fyf_y are measured in pixels, rather than millimeters. The principal point (cx,cy)(c_x,c_y) is where the optical axis intersects the image plane; it is often near the image center but need not equal it. The explorer fixes it at (320, 240) and sets both focal lengths to the same value. The zero-skew intrinsic matrix is:

K=[fx0cx0fycy001].K = \begin{bmatrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}.

With Pc=(1,1,4)TP_c=(1,1,4)^T, fx=fy=400f_x=f_y=400 and principal point (320, 240), the normalized position is (0.25, 0.25) and the projected pixel is (420, 340). Doubling the focal lengths changes it to (520, 440): the offset from the principal point doubles. The 3D point and its depth stay fixed.

For a fixed sensor resolution, a larger focal length gives a narrower field of view. Resizing an image also changes its pixel calibration: scaling both dimensions by a factor scales the focal lengths and principal point by that factor. Cropping shifts the principal point by the crop's origin. These are changes in the pixel coordinate system, not changes in the scene. OpenCV documents intrinsic parameters and image scaling.

The explorer uses exact calculations on an authored point, with world Y fixed at 0.5 scene units. Start by moving the camera right, then change the focal length. Choose Outside and Behind to compare a point beyond the image edge with one that has negative camera depth. Same ray adds a second point twice as far from the camera, including when the camera is translated or rotated.

From a world point to a pixel
Preset
Points
4.0 units
400 px
0.00 units
0°

World top view: +Z right, +X down

Square: camera center C. Arrow: camera forward axis. Teal: P. World grid spacing: 2 scene units.

Image: u right, v down · 640 × 480 pixels

Cross: principal point (320, 240). Teal dot: projected point.

Exact calculation of an authored scene. From Start, move the camera right: the fixed world point moves left in the image. Increase focal length: its offset from the principal point grows.

A projected location is not a visibility test

Three cases need separate treatment:

  • Positive camera depth and inside image bounds: a candidate image location exists. A nearer surface may still occlude the point.
  • Positive camera depth and outside image bounds: the point projects beyond the recorded image. It remains in front of the camera.
  • Zero or negative camera depth: depth division is undefined at zero, and a negative-depth point is behind the forward-looking camera. Its algebraic ratio is not an observation in this camera's image.

The explorer checks the continuous coordinate bounds 0≤u<6400 \le u < 640 and 0≤v<4800 \le v < 480, without rounding to an array element. It does not render surfaces or test occlusion. Near-zero camera depths are also excluded from projection to avoid unstable division.

One image point leaves depth undetermined

Suppose the calibration is known and a pixel has been observed. Reversing the intrinsic conversion gives a camera direction, but not a distance:

Pc=Zc[(u−cx)/fx(v−cy)/fy1],Zc>0.P_c = Z_c\begin{bmatrix} (u-c_x)/f_x \\ (v-c_y)/f_y \\ 1 \end{bmatrix}, \qquad Z_c > 0.

Every positive choice of ZcZ_c produces a point on the same ray. In the world frame that ray is Pw=C+RTPcP_w=C+R^T P_c. The explorer's amber point scales the displacement from the camera center by two, so both points project to the same pixel even after changing the camera pose.

A single calibrated image point therefore cannot determine a unique 3D position. A second view, a known object size, a depth sensor, or additional scene assumptions can supply information that the projection discarded. A learned depth estimate also relies on information beyond this one point's coordinates.

Where the ideal model stops

Real lenses can bend straight lines through distortion. In a calibrated distorted-camera model, normalized coordinates are distorted before focal scaling and principal-point offsets are applied. The explorer uses zero distortion, zero skew and a fixed principal point; it is an illustration of projection, not a camera calibration or reconstruction result. OpenCV describes radial and tangential distortion alongside the ideal model.

Projection explains how camera motion and scene depth can change image positions. Optical flow next asks which image location corresponds to a point in another frame. Flow can be estimated directly in 2D; knowing the camera geometry helps interpret it, but a flow vector alone does not uniquely reveal 3D motion or depth.

If you found this explanation helpful, consider sharing it with others.

Mastodon