Skip to main content

Focal Loss: Focusing on Hard Examples

Learn focal loss for deep learning: down-weight easy examples, focus on hard ones. Interactive demos of gamma, alpha balancing, and RetinaNet.

Focal Loss: Focusing on Hard Examples

Focal loss addresses a fundamental problem in classification: when easy examples vastly outnumber hard ones, standard cross-entropy loss is dominated by the easy majority. The model spends most of its gradient budget reinforcing what it already knows instead of learning from its mistakes.

Introduced in the 2017 RetinaNet paper by Lin et al., focal loss adds a simple modulating factor to cross-entropy that automatically down-weights the contribution of easy examples and focuses training on hard negatives. This single change enabled one-stage object detectors to match the accuracy of two-stage detectors for the first time, while running significantly faster.

The Teacher Analogy

Think of a classroom with 100 students. Ninety-five students ace every quiz, while five consistently struggle. A teacher using standard cross-entropy spends equal effort grading every student. But an effective teacher would notice that the 95 high-achievers need almost no attention and redirect all effort toward the five who need help. This is exactly what focal loss does: it measures each example's confidence, then assigns proportionally less loss to confident (easy) predictions and more to uncertain (hard) ones.

With a higher gamma, almost all effort goes to the hardest cases, risking neglect of borderline cases that remain somewhat uncertain. In object detection, the imbalance is like 100,000 easy background examples drowning out 10 actual objects. The Hard/Easy ratio compares the total attention shares of the two groups.

The quiz below uses seven students: five easy students with confidence above 88%, and two hard students with confidence below 35%. Press Play to reveal how much attention each receives.

Aggregate loss attention
Loss
GroupCorrectLoss shareShare
A ×10095%
—
B ×10092%
—
C ×10088%
—
D ×10097%
—
E ×10091%
—
F ×135%
—
G ×122%
—
500 easy examples A–E2 hard examples F–G

No groups revealed.

Authored probabilities: 500 easy examples and 2 hard examples. Bars represent summed loss contributions, not gradient norms or measured training loss share.

Mathematical Definition

Standard Cross-Entropy

Helpful background

Cross-entropy loss

Connect probability assigned to the target with the classification loss.

Open full page

For binary classification, the standard cross-entropy loss is:

CE(p, y) = -\log(p_t)

where p_t is defined as p when the ground-truth class is 1, and 1 - p when the class is 0. In other words, p_t is the model's estimated probability for the correct class. A well-classified example has p_t \to 1 and incurs a small loss; a misclassified one has p_t \to 0 and incurs a large loss.

The Focal Loss Modification

Focal loss multiplies the cross-entropy by a modulating factor:

FL(p_t) = -(1 - p_t)^\gamma \log(p_t)

When p_t is large (the model is confident and correct), the factor (1 - p_t)^\gamma shrinks toward zero, dramatically reducing the loss. When p_t is small (the model is wrong or uncertain), the factor stays near 1, and the loss is essentially unchanged from cross-entropy. The parameter \gamma \geq 0 controls how aggressively easy examples are down-weighted. Setting \gamma = 0 recovers standard cross-entropy.

The Alpha-Balanced Variant

In practice, focal loss is usually combined with a class-balancing weight \alpha_t:

FL(p_t) = -\alpha_t (1 - p_t)^\gamma \log(p_t)

Here \alpha_t serves a different role than \gamma. While gamma handles the difficulty imbalance (easy vs hard examples), alpha handles the frequency imbalance (common vs rare classes). The original RetinaNet paper found that using both together yielded the best results, with \alpha = 0.25 and \gamma = 2 as optimal defaults on the COCO benchmark.

Adjust gamma to see how focal loss reshapes the loss curve compared to standard cross-entropy. Click the chart, or use Inspect probability, to inspect exact values; the dashed reference curve can be toggled.

Gamma near zero provides virtually no down-weighting. Mild focusing reduces easy losses but leaves a modest gap between easy and hard losses. The recommended range for most tasks reaches gamma=2; strong focusing can suit extreme imbalance, while very aggressive focusing risks instability when the hardest examples include label noise or outliers.

Loss curves
2.0
0.50
Focal loss curve for gamma 2.0 with the cross-entropy curve, loss against probability of the correct class; click to inspect probability
Focal Loss (γ=2.0)
Cross-Entropy (γ=0), dashed
Loss reduction

Recommended range for most tasks.

The Gamma Effect

The focusing parameter \gamma is the heart of focal loss. Consider what happens to the modulating factor (1 - p_t)^\gamma at different confidence levels:

For an easy example with p_t = 0.9 and \gamma = 2, the factor equals (1 - 0.9)^2 = 0.01, reducing the loss by 100x. For a hard example with p_t = 0.2, the factor is (1 - 0.2)^2 = 0.64, keeping most of the original loss. This asymmetry is precisely the mechanism that shifts the model's attention from easy backgrounds to hard objects.

As gamma increases, the suppression of easy examples becomes more extreme. At \gamma = 5, the easy example's loss is reduced by a factor of 100,000, essentially zeroing it out. The model trains almost exclusively on examples it finds difficult.

Compare each example's share of the total loss under cross-entropy and focal loss. Choose a gamma preset to see paired contributions for five difficulty levels. The easy share covers the first two examples, and the hard share covers the last two; the Medium example is excluded from both group readouts. Hard focus boost divides the focal-loss hard share by the cross-entropy hard share.

At gamma=0, easy examples consume loss budget despite contributing little useful learning signal. A small gamma shifts the budget mildly, so easy examples still take a disproportionate share. At gamma=2, hard examples dominate, forcing improvement of the weakest predictions. Higher gamma makes each step highly targeted, but extreme focusing risks sensitivity to noisy hard labels.

Loss budget
Gamma

Share of Total Loss

Very Easyp=0.95
CE
1.2%
FL
0.0%
Easyp=0.85
CE
3.8%
FL
0.1%
Mediump=0.6
CE
12.1%
FL
3.2%
Hardp=0.3
CE
28.5%
FL
23.2%
Very Hardp=0.1
CE
54.4%
FL
73.4%
0%42%84%
Cross-Entropy (γ=0)
Focal Loss (γ=2)

RetinaNet’s COCO optimum.

Alpha Balancing for Imbalanced Classes

While gamma addresses difficulty imbalance (easy vs hard), many real datasets also suffer from frequency imbalance (many negatives, few positives). In dense object detection, a typical image produces over 100,000 anchor boxes, of which fewer than 10 overlap with actual objects. Even after focal loss suppresses easy negatives, the sheer number of negative examples can still overwhelm the positive ones.

The alpha parameter provides frequency balancing. For the positive class, the loss is multiplied by \alpha, and for the negative class by 1 - \alpha. A common heuristic is to set alpha inversely proportional to class frequency, giving the rare positive class a higher weight. The RetinaNet paper found \alpha = 0.25 optimal, which may seem counterintuitive since positives are rare, but gamma already suppresses easy negatives so aggressively that a smaller alpha for positives turns out to work best.

Choose a dataset to reset alpha to the inverse-frequency recommendation while retaining gamma, or use Auto to apply the current recommendation. The balanced preset has equal positive and negative counts and needs no alpha balancing; moderate imbalance is a 1:10 ratio common in medical imaging and fraud detection. Severe imbalance is a 1:100 ratio typical of dense detection with mostly background anchors; extreme imbalance is 1:10,000, as in RetinaNet with fewer than 10 objects among 100,000 candidates.

The simulated batch uses five positive and five negative confidences. Average weighted loss is multiplied by each class's frequency to obtain its effective contribution, then divided by the total contribution for its loss share. Low alpha gives positives less weight, alpha=0.50 gives equal weights, and high alpha gives positives more weight. Gamma=0 removes focusing, gamma=2 is the default, and gamma=5 is extreme. Roughly balanced loss shares indicate that alpha and gamma work together effectively. When the positive class dominates despite being rare, lower alpha or let it auto-calibrate; when negatives still dominate, increase alpha. For partial imbalance, move closer to the recommendation.

Class balancing
Dataset
0.25
2.0
Positive Examples
10
α = 0.25 weight
33.2% loss share
Negative Examples
1,000
(1-α) = 0.75 weight
66.8% loss share
Positive loss shareNegative loss share

Some imbalance remains; balance the displayed loss.

Calculated from authored confidence samples. Auto chooses α = negative unweighted loss / (positive + negative unweighted loss), balancing this display exactly. Equal loss shares are an illustrative target, not an optimal training recommendation.

Comparing Imbalanced Learning Strategies

Focal loss is not the only approach to class imbalance. Weighted cross-entropy adjusts loss by class frequency but ignores difficulty. Online Hard Example Mining (OHEM) keeps only the hardest examples per batch but discards the rest entirely. Class-Balanced Loss uses the effective number of samples to derive theoretically motivated weights but does not adapt to per-example difficulty.

Focal loss occupies a unique position: it is differentiable, handles difficulty natively through the modulating factor, and requires only two additional hyperparameters. It adds virtually no computational cost compared to standard cross-entropy since the modulating factor involves only a single exponentiation.

Compare frequency correction, difficulty awareness, speed and simplicity across the five strategies, including each method's best use.

Imbalanced learning strategies

Cross-Entropy

Standard log-loss for classification

Frequencypoor
No frequency correction
Difficultypoor
Equal weight for easy and hard
Speedexcellent
Minimal overhead
Simplicityexcellent
None beyond learning rate

Best for: Balanced datasets, baseline training

Weighted CE

Class weights inversely proportional to frequency

Frequencyexcellent
Direct frequency compensation
Difficultypoor
No difficulty awareness
Speedexcellent
Minimal overhead
Simplicitymoderate
Per-class weights to tune

Best for: Moderate imbalance, known class frequencies

Focal Loss

Down-weights easy examples via modulating factor

Frequencymoderate
Alpha provides partial balancing
Difficultyexcellent
Gamma auto-focuses on hard cases
Speedexcellent
Negligible extra cost
Simplicitymoderate
Alpha and gamma to tune

Best for: Dense object detection, extreme imbalance

Class-Balanced Loss

Uses effective number of samples for weighting

Frequencyexcellent
Theoretically grounded weighting
Difficultymoderate
Can combine with focal term
Speedexcellent
Only needs class counts
Simplicitymoderate
Beta parameter for effective samples

Best for: Long-tailed recognition, many classes

OHEM

Trains only on hardest examples per batch

Frequencymoderate
Indirect via hard example selection
Difficultyexcellent
Explicitly selects hardest examples
Speedmoderate
Sorting overhead per batch
Simplicitymoderate
Keep ratio to tune

Best for: Two-stage detectors, when batch sorting is cheap

Use focal loss when:

  • Extreme class imbalance (1:1000+)
  • Dense object detection or segmentation
  • Easy negatives dominate gradient signal
  • A simple, effective baseline is needed

Consider alternatives when:

  • Long-tailed distribution with many classes
  • Label noise is present in hard examples
  • Theoretically grounded class weights are needed
  • Two-stage detectors where OHEM is natural

When to Use Focal Loss

Dense object detection is the canonical use case. One-stage detectors like RetinaNet, FCOS, and ATSS all use focal loss to handle the extreme foreground-background imbalance inherent in processing every location in a feature map. Without focal loss, the easy background examples overwhelm the gradient and the detector fails to learn foreground patterns.

Medical imaging frequently involves severe class imbalance. Diseases like diabetic retinopathy or rare tumors may represent fewer than 1% of samples. Focal loss helps the model focus on the subtle features that distinguish pathological from healthy tissue rather than simply predicting the majority class.

Semantic segmentation benefits when certain classes occupy far fewer pixels than others. Roads and buildings may dominate a scene while pedestrians and signs are small and rare. Focal loss ensures the network learns to segment these minority classes accurately.

Any classification task with severe imbalance where easy examples dominate can benefit. Fraud detection, defect inspection, and anomaly detection all fit this pattern.

Common Pitfalls

1. Training Instability at Initialization

The most dangerous pitfall is failing to initialize the classification head properly. At the start of training, the model predicts roughly equal probability for all classes. In an extreme imbalance scenario (1:10,000), this means the model initially assigns roughly 50% probability to the positive class even though positives represent 0.01% of examples. The focal loss for this mass of confident-but-wrong negatives produces enormous gradients that destabilize training.

The solution, used in RetinaNet, is to initialize the final layer's bias so that the model predicts the positive class with a very low prior probability (typically 0.01). This way, negatives start as easy (correctly classified) and contribute little loss, preventing the initial gradient explosion.

2. Sensitivity to Label Noise

Because focal loss concentrates all learning signal on the hardest examples, label noise in those hard examples has an outsized effect. A mislabeled example will always appear "hard" to the model, and focal loss will persistently amplify its contribution to the gradient. If your dataset contains significant label noise, consider using a smaller gamma or combining focal loss with label smoothing.

3. Interaction Between Alpha and Gamma

Alpha and gamma are not independent. Changing gamma shifts the effective weight distribution across examples, which changes the optimal alpha. The RetinaNet paper systematically searched a grid of alpha and gamma values and found they must be tuned together. A common starting point is \gamma = 2 with \alpha = 0.25, but these should be validated on your specific dataset.

4. Not Always Necessary

For balanced or mildly imbalanced datasets, focal loss may not improve over standard cross-entropy. The down-weighting of easy examples reduces the effective training signal, which can slow convergence when there is no imbalance problem to solve. Always compare against a vanilla cross-entropy baseline.

If you found this explanation helpful, consider sharing it with others.

Mastodon