Overview
Modern object detection has evolved from complex multi-stage pipelines (R-CNN family) and anchor-based single-shot detectors (YOLO, SSD) to elegant transformer-based architectures. DETR (DEtection TRansformer) pioneered this shift by treating object detection as a direct set prediction problem, eliminating hand-designed components like anchor boxes, non-maximum suppression (NMS), and region proposal networks.
The key innovation is using learned object queries to distinguish decoder prediction slots that attend to image features. Combined with bipartite matching during training, DETR predicts a set of detections without an NMS duplicate-suppression stage. Scores and normalized box coordinates still need interpretation for evaluation or display.
DETR Architecture Walkthrough
Step through DETR's pipeline one stage at a time — from raw pixels to a set of predictions — watching the tensor shapes and operation at each stage. A single forward pass replaces the multi-stage anchor-and-NMS machinery of earlier detectors.
- Stage
- 1 / 6
- Tensor shape
- B×3×800×1066
- This stage
- raw pixels
- Operation
- Image input
Input Image
H × W × 3 (e.g., 800 × 1066 × 3)
Input Processing
- RGB image of arbitrary size
- Resized with max dimension 1333px
- Normalized with ImageNet mean/std
- Batched with padding if needed
# No region proposals needed! # No anchor boxes! # Just raw image pixels
Stage 1 of 6: Input Image
DETR's Key Innovation: By treating object detection as a set prediction problem with bipartite matching, DETR eliminates hand-designed components like anchor boxes, NMS, and region proposals — achieving a truly end-to-end detection pipeline.
Key Concepts
Object Queries
100 learned embeddings that specialize to detect different objects, positions, and scales during training
Set-Based Prediction
Predict all objects simultaneously without sequential decoding or duplicate suppression
Bipartite Matching
Hungarian algorithm optimally assigns predictions to ground truth during training
Cross-Attention
Object queries attend to encoder features to gather visual information for detection
End-to-End Training
No anchor boxes, NMS, or region proposals - just image in, detections out
Global Reasoning
Transformer self-attention enables reasoning about all objects and their relationships
Object Query Attention Visualization
Pick a query and scrub through an authored illustration of cross-attention sharpen — from a diffuse scan of the whole image into a focused lock on a single object.
Decoder layer 1 of 6
- Query
- #0
- Predicts
- person
- Confidence
- 95%
- Decoder layer
- 1 / 6
- Hovered cell
- —
- Attention
- —
Authored illustration: The attention weights, classes, confidence scores, and boxes are invented; no model or image was evaluated. The broad-to-sharp layer schedule is a chosen construction, not a measured result.
Illustrated Attention Evolution Through Layers
Early layers (1-2): The authored map is broad and diffuse.
Middle layers (3-4): The authored map becomes more focused.
Late layers (5-6): The authored map is sharper around its chosen focus.
Understanding Object Queries
Helpful background
Object queries
Distinguish learned prediction slots from image tokens and attention Q, then follow how decoder states produce detections.
Object queries are learned slot embeddings. In the original implementation, decoder content states start at zero and the learned embeddings provide query-position information. Cross-attention lets each slot read image features, while self-attention lets the evolving slot states exchange information.
Each final slot state produces class scores and a box. Slots can develop spatial or scale preferences, but they have no fixed class identity. One-to-one matching and the set loss train distinct predictions; self-attention alone does not guarantee uniqueness. Unmatched slots receive the no-object classification target. Original decoder, prediction heads and training criterion.
Bipartite Matching for Training
Step through how the Hungarian algorithm pairs DETR's predictions with the ground-truth objects, turning detection into a one-to-one assignment problem that needs no duplicate suppression.
Initial State
DETR outputs N=100 predictions (showing 8 for clarity)
Authored illustration: classes, probabilities, and normalized boxes are invented. All eight slots enter the cost matrix, including those currently predicting no object. Cost = −p(target class) + 5 × L1 (center, width, height) − 2 × GIoU; this matching cost is distinct from the training loss.
- Step
- 1 / 5
- Matched
- —
- Unmatched
- —
- Total cost
- —
Predictions (N=100, showing 8)
- Q0persontop class probability: 95%target probabilities: person 0.95, dog 0.01, car 0.01; box [0.12, 0.15, 0.38, 0.88]
- Q1dogtop class probability: 87%target probabilities: person 0.02, dog 0.87, car 0.02; box [0.52, 0.48, 0.78, 0.92]
- Q2cartop class probability: 91%target probabilities: person 0.02, dog 0.02, car 0.91; box [0.58, 0.08, 0.95, 0.42]
- Q3persontop class probability: 23%target probabilities: person 0.23, dog 0.1, car 0.1; box [0.7, 0.6, 0.85, 0.95]
- Q4cattop class probability: 15%target probabilities: person 0.1, dog 0.1, car 0.1; box [0.1, 0.7, 0.3, 0.9]
- Q5birdtop class probability: 8%target probabilities: person 0.04, dog 0.04, car 0.04; box [0.4, 0.2, 0.5, 0.35]
- Q6no objecttop class probability: 92%target probabilities: person 0.02, dog 0.02, car 0.02; box [0.05, 0.05, 0.2, 0.2]
- Q7no objecttop class probability: 88%target probabilities: person 0.03, dog 0.03, car 0.03; box [0.8, 0.75, 0.95, 0.95]
Ground Truth (M=3)
- G0personbbox: [0.14, 0.18, 0.36, 0.85]
- G1dogbbox: [0.54, 0.50, 0.76, 0.90]
- G2carbbox: [0.60, 0.10, 0.93, 0.44]
Key Insight: This exhaustive solver finds the global minimum for eight authored slots; DETR uses linear assignment. Each ground truth is assigned to exactly one prediction, enabling set-based training without NMS or anchor boxes.
Bipartite Matching Loss
Helpful background
Set prediction and matching
Find a one-to-one assignment with minimum total cost and distinguish training assignment from detection evaluation and NMS.
Helpful background
NMS and Soft-NMS
Use IoU to follow duplicate suppression and compare removing boxes with decaying their scores.
Traditional detectors assign multiple predictions to each ground truth (via IoU thresholds), then suppress duplicates with NMS. DETR takes a fundamentally different approach:
- Cost Matrix: Compute pairwise costs between all predictions and ground truth objects
- Hungarian Algorithm: Find optimal 1-to-1 assignment minimizing total cost
- Loss Computation: Matched pairs contribute class and box losses; unmatched slots contribute no-object classification loss
The cost combines classification probability and bounding box distance (GIoU + L1):
Helpful background
Box regression and GIoU
Compare coordinate losses with overlap-based objectives and inspect the enclosure term when boxes do not overlap.
L_match(y_i, ŷ_σ(i)) = -1_{c_i≠∅} p̂_σ(i)(c_i) + 1_{c_i≠∅} L_box(b_i, b̂_σ(i))
Real-World Applications
Autonomous Driving
End-to-end detection of vehicles, pedestrians, and obstacles without complex post-processing
Medical Imaging
Detect tumors, lesions, and anatomical structures in X-rays, CT, and MRI scans
Satellite & Aerial Imagery
Detect buildings, vehicles, and infrastructure across large-scale images
Robotics & Manipulation
Real-time object detection for grasping and navigation
Video Understanding
Track objects across frames using query propagation
Document Analysis
Detect text regions, tables, and figures in documents
This teaching sketch uses learned embeddings directly as decoder content and adds image positions once before the transformer. Original DETR instead starts decoder content at zero and injects image and query positional embeddings into attention queries and keys throughout its layers. The sketch illustrates the feature-to-prediction pipeline; it does not reproduce the original decoder. See the official transformer implementation for those details.
Simplified DETR Implementation
import torch
import torch.nn as nn
from torchvision.models import resnet50
class SimpleDETR(nn.Module):
def __init__(self, num_classes=91, num_queries=100, hidden_dim=256):
super().__init__()
# CNN Backbone (ResNet-50, remove final layers)
backbone = resnet50(pretrained=True)
self.backbone = nn.Sequential(*list(backbone.children())[:-2])
self.conv = nn.Conv2d(2048, hidden_dim, 1) # Reduce channels
# Transformer
self.transformer = nn.Transformer(
d_model=hidden_dim,
nhead=8,
num_encoder_layers=6,
num_decoder_layers=6,
dim_feedforward=2048,
batch_first=True
)
# Object queries (learned embeddings)
self.query_embed = nn.Embedding(num_queries, hidden_dim)
# Prediction heads
self.class_head = nn.Linear(hidden_dim, num_classes + 1) # +1 for "no object"
self.bbox_head = nn.Sequential(
nn.Linear(hidden_dim, hidden_dim),
nn.ReLU(),
nn.Linear(hidden_dim, hidden_dim),
nn.ReLU(),
nn.Linear(hidden_dim, 4),
nn.Sigmoid() # Normalized coordinates
)
# Positional encoding
self.pos_encoder = PositionalEncoding2D(hidden_dim)
def forward(self, x):
# Extract features: (B, 3, H, W) -> (B, 2048, H/32, W/32)
features = self.backbone(x)
features = self.conv(features) # (B, 256, H/32, W/32)
B, C, H, W = features.shape
# Add positional encoding
pos = self.pos_encoder(H, W).to(x.device)
features = features + pos
# Flatten spatial dimensions: (B, 256, H*W) -> (B, H*W, 256)
src = features.flatten(2).permute(0, 2, 1)
# Object queries: (num_queries, 256) -> (B, num_queries, 256)
queries = self.query_embed.weight.unsqueeze(0).repeat(B, 1, 1)
# Transformer: encode image, decode with queries
output = self.transformer(src, queries) # (B, num_queries, 256)
# Predictions
class_logits = self.class_head(output) # (B, 100, num_classes+1)
bbox_pred = self.bbox_head(output) # (B, 100, 4)
return {'pred_logits': class_logits, 'pred_boxes': bbox_pred}
class PositionalEncoding2D(nn.Module):
def __init__(self, hidden_dim, temperature=10000):
super().__init__()
if hidden_dim <= 0 or hidden_dim % 4:
raise ValueError("hidden_dim must be a positive multiple of 4")
self.hidden_dim = hidden_dim
self.temperature = temperature
def forward(self, h, w):
y_embed = torch.arange(h).unsqueeze(1).repeat(1, w)
x_embed = torch.arange(w).unsqueeze(0).repeat(h, 1)
dim_t = torch.arange(self.hidden_dim // 4)
dim_t = self.temperature ** (2 * dim_t / (self.hidden_dim // 2))
pos_x = x_embed.unsqueeze(-1) / dim_t
pos_y = y_embed.unsqueeze(-1) / dim_t
pos_x = torch.stack([pos_x.sin(), pos_x.cos()], dim=-1).flatten(-2)
pos_y = torch.stack([pos_y.sin(), pos_y.cos()], dim=-1).flatten(-2)
pos = torch.cat([pos_y, pos_x], dim=-1)
return pos.permute(2, 0, 1).unsqueeze(0) # (1, hidden_dim, H, W)
# Usage
model = SimpleDETR(num_classes=91)
image = torch.randn(2, 3, 800, 1066)
output = model(image)
print(f"Class predictions: {output['pred_logits'].shape}") # (2, 100, 92)
print(f"Box predictions: {output['pred_boxes'].shape}") # (2, 100, 4)Advantages & Limitations
Advantages
- ✓End-to-end training without hand-crafted components (anchors, NMS)
- ✓Global reasoning via self-attention handles object relationships
- ✓Excellent performance on large objects due to global context
- ✓Simpler pipeline with fewer hyperparameters to tune
- ✓Easily extended to panoptic segmentation and tracking
- ✓Set-based prediction naturally handles varying object counts
Limitations
- ×Slow training convergence (500 epochs vs 36 for Faster R-CNN)
- ×Struggles with small objects due to coarse feature resolution
- ×High memory usage from full attention over image features
- ×Fixed number of queries limits maximum detections per image
- ×Requires careful learning rate scheduling and data augmentation
- ×Inference slower than optimized CNN-based detectors
Evolution: Deformable DETR and Beyond
The original DETR's limitations sparked rapid innovation:
The following published COCO val2017 small-object AP results use different training schedules and configurations, so they are not controlled comparisons.
| Model and configuration | Epochs | Small-object AP | Source |
|---|---|---|---|
| DETR, ResNet-50 | 500 | 20.5 | Deformable DETR, Table 1 |
| Deformable DETR, ResNet-50, multi-scale | 50 | 26.4 | Deformable DETR, Table 1 |
| DINO, ResNet-50, 4 scales | 12 | 32.0 | DINO, Table 1 |
| RT-DETR, ResNet-50 | 72 | 34.8 | RT-DETR, Table 2 |
Conditional DETR introduces conditional cross-attention, while DAB-DETR uses anchor-based queries.
Deformable DETR is particularly impactful: it replaces full attention with deformable attention that only attends to a small set of key sampling points, reducing complexity from O(N²) to O(N).
Best Practices
- Use Deformable Attention: For practical applications, use Deformable DETR or its variants for faster training and better small object detection
- Multi-Scale Features: Use FPN-style multi-scale features to capture objects at different sizes
- Auxiliary Losses: Add prediction heads after each decoder layer for faster convergence
- Strong Augmentation: Use aggressive data augmentation (random crop, scale jitter, color jitter) to compensate for slow convergence
- Careful Learning Rate: Use lower learning rate for backbone (1e-5) and higher for transformer (1e-4)
- Query Design: Consider anchor-based queries (DAB-DETR) or denoising training (DINO) for faster convergence
Mathematical Foundation
The detection loss combines classification and localization:
L(y, ŷ) = Σ_i [ λ_cls · L_cls(c_i, ĉ_σ(i)) + 1_{c_i≠∅} · (λ_L1 · ||b_i - b̂_σ(i)||_1 + λ_giou · L_giou(b_i, b̂_σ(i))) ]
Where:
σis the optimal assignment from Hungarian matchingL_clsis classification cross-entropy, with a lower weight for the no-object classL_giouis generalized IoU loss for better box regression- The loss weights are configuration choices; the original released loss uses unit classification weight,
λ_L1=5andλ_giou=2. The no-object classification term has its own relative weight.
The Hungarian matching cost:
C_match(y_i, ŷ_j) = -p̂_j(c_i) + λ_L1 · ||b_i - b̂_j||_1 + λ_giou · L_giou(b_i, b̂_j)
Further Reading
For the sampling mechanism behind Deformable DETR, start with deformable attention and reference points, then continue to the paper walkthrough.
- DETR: End-to-End Object Detection with Transformers - Original paper
- Deformable DETR - Efficient attention for detection
- DAB-DETR - Dynamic anchor boxes as queries
- DINO - Denoising training for faster convergence
- RT-DETR - Real-time transformer detection
Related concepts
Follow a query from a spatial reference through learned offsets, bilinear feature sampling and weighted aggregation, with exact non-square coordinate arithmetic.
Follow spatial image features into DETR decoder slots, separating learned query embeddings, attention queries, changing content and object predictions.
Compare anchor-based vs anchor-free object detection: Faster R-CNN and RetinaNet anchors vs FCOS and CenterNet point-based methods.
Learn how DETR assigns unordered object targets to prediction slots, why greedy choices can fail, and how matching differs from training loss and evaluation.
Learn how normalized box coordinates, L1 loss, IoU, and Generalized IoU guide object localization, including disjoint boxes and containment limits.
Learn how Feature Pyramid Networks build multi-scale feature representations through top-down pathways and lateral connections for robust object detection.
