Skip to main content

Focal Loss for Dense Object Detection

Separate alpha weighting from focal modulation, derive the complete logit gradient, and inspect population imbalance, prior initialization and RetinaNet evidence.

TL;DR

  • Focal loss discounts confidently correct decisions so that many easy background examples contribute less to a dense detector's classification objective. It changes difficulty weighting within a class, while alpha supplies a separate class multiplier.
  • With alpha present, gamma zero gives alpha-weighted binary cross entropy. For positive gamma, the focusing factor also depends on the logit and must be differentiated.
  • Population counts still matter: many discounted examples can retain a large total contribution, and persistently hard examples may include annotation errors. Stronger focusing is not universally better.
  • RetinaNet's reported gains belong to specified detector and training recipes, including a rare-foreground initialization prior. The curves and standard-library reference here check loss arithmetic and gradients rather than reproducing detector AP.

Why easy negatives dominate

A dense detector makes an enormous number of binary classification decisions. Most candidate locations are background, and many quickly become easy background examples. A tiny loss at each of those locations can still add up to a large share of the objective. Focal loss changes that accounting: it discounts examples according to how confidently the model already predicts their correct outcome.

This review follows Focal Loss v2, the February 2018 revision of the ICCV 2017 paper. The displayed curves, gradients and population counts are analytically calculated teaching examples, not recorded RetinaNet training statistics. Original study results are explicitly cited. The standard-library reference does not download a checkpoint or measure GPU performance.

Imbalance has a count and a difficulty distribution

Imagine ten thousand negative decisions, each with a loss of only one hundredth, alongside a much smaller number of uncertain objects. The negative sum can dominate even though no individual negative is especially difficult. An optimizer sees the aggregate objective, not a human judgment that the easy examples are already solved.

A fixed positive/negative class weight can change the relative contribution of the two groups. It cannot distinguish an easy negative from a hard negative within the same group. Sampling can reduce how many examples enter a training step. Focal loss instead keeps a confidence-dependent factor inside the objective. These are different interventions and can interact.

The detector setting matters. A two-stage system filters candidates through proposals and can sample foreground/background regions. A dense one-stage system makes predictions across many positions, scales and anchors directly. The focal-loss paper studies whether reshaping the classification objective allows such a detector to train effectively under that imbalance. Original paper, §§1–3

This is narrower than the claim that all imbalanced learning problems require focal loss. Different annotation errors, score distributions, matching rules and metrics can produce different tradeoffs. A loss changes what mistakes receive emphasis; it does not establish that all highly emphasized examples are useful or correctly labeled.

Start with the probability of the correct outcome

Let p be a sigmoid's probability for the positive class. For binary target y∈1, define pₜ=p when y=1 and pₜ=1−p when y=0. Thus pₜ always means the probability assigned to the observed outcome. A confidently correct background example has a large pₜ even though its positive-class probability p is small.

Ordinary binary cross entropy can then be written −log pₜ. Alpha balancing uses αₜ=α for positives and αₜ=1−α for negatives. This convention must be stated: alpha is the positive-class weight, not automatically the minority-class weight in every labeling scheme. The alpha-balanced focal form is:

\operatorname{FL}(p_t)=-\alpha_t(1-p_t)^\gamma\log(p_t),\qquad\gamma\ge0

The paper first introduces the unweighted modulation and then the alpha-balanced version used in its experiments. Keeping the class weight separate helps interpret limiting cases. Original paper, Equations 1–5

Discount easy examples continuously

The focusing factor depends on pₜ, the probability assigned to the ground-truth binary outcome. Alpha weighting is separate from this confidence-based discount.

0.9
2
0.25

Loss before class weighting (αₜ = 1)

50

Ground-truth probability pₜ: 0 → 1

factor (1−pₜ)^γ = 0.010000
weighted BCE = 0.026340
focal loss = 0.000263

Calculated loss curve, not observed RetinaNet training. The curve omits alpha to isolate gamma; the numeric readout includes the selected class weight.

At γ=0 the focusing factor is one. When alpha is present, the result is alpha-weighted BCE. It becomes unweighted BCE only when the class multiplier is removed or set uniformly to one. Describing every gamma-zero implementation as ordinary BCE hides a potentially large difference in class weighting.

For γ=2 and pₜ=0.9, the focusing factor is 0.01. The focal loss is therefore one hundredth of the corresponding alpha-weighted BCE at that same probability. At pₜ=0.99, the factor is 0.0001. These are exact ratios between two loss values, not claims of one-hundredfold faster training or an equivalent reduction in gradient magnitude.

When pₜ is near zero, the factor is near one. Difficult examples retain much of their cross-entropy contribution while easy ones are discounted. No hard threshold decides that an example is ignored, and a finite-confidence easy example normally retains a nonzero loss. A sufficiently large number of those examples can still matter.

Differentiate the modulation too

A subtle mistake is to multiply the BCE gradient by the focusing factor while treating that factor as a constant. But pₜ depends on the model's logit, so the factor itself changes when the logit changes. The product rule contributes another term.

For positive-class logit z and sign s=2y−1, the probability pₜ is sigmoid(sz). Differentiating the complete focal loss gives:

\frac{\partial L}{\partial z}=\alpha_t s(1-p_t)^\gamma\left[\gamma p_t\log p_t-(1-p_t)\right]

This derivative is calculated here from the paper's objective. The source crop supports the loss definition; it is not presented as a derivative printed in the original paper. The numerical reference checks it against central finite differences for both labels, several logits and fractional as well as integer gamma values.

The focusing factor also has a derivative

Multiplying the BCE gradient by (1−pₜ)^γ treats the factor as constant. Ordinary focal loss differentiates the factor as well.

0
2

Full derivative dFL/dz

-0.0745717

Detached factor × BCE gradient

-0.0312500

pₜ = 0.50000; αₜ = 0.25. Missing focusing-factor derivative = -0.0433217. At γ=0 both reduce to the same alpha-weighted BCE derivative.

The exact logit derivative is checked against central finite differences in both JavaScript and Python. The detached value is a counterexample, not the derivative of the displayed loss.

At γ=0 the extra product-rule term vanishes, leaving the expected alpha-weighted BCE derivative. For γ>0, treating the focusing factor as detached changes the gradient and therefore the optimization procedure. A framework implementation that accidentally detaches it can still return the correct displayed scalar loss while training a different update rule.

The sign is also instructive. A positive target generally needs the positive-class logit pushed upward, so its loss derivative is negative. A negative target has the opposite sign. Swapping p and pₜ for negatives can accidentally focus on easy foreground confidence instead of correct-background confidence. Tests across both labels catch this mistake.

Very confident predictions create numerical issues if probabilities are rounded to exactly zero or one before logarithms. The reference computes log pₜ through a stable softplus expression and obtains 1−pₜ using expm1. It avoids arbitrary probability clipping, which can change gradients around the clip boundary. Extreme finite logits are tested for finite loss and derivatives.

Count the population, not just one example

The loss curve describes one decision. Training uses a collection of decisions whose counts and difficulties evolve. To see that distinction, hold three groups at supplied probabilities: many easy negatives, one hundred hard negatives and twenty positives. Multiply each per-example loss by the group count.

Many small losses can still dominate

An authored population contains many easy negatives, 100 hard negatives and 20 positives. Counts and probabilities are supplied independently; this is not a recorded anchor distribution.

10000
0.99
2
GroupCountpₜLoss sumΣ |dL/dz|
Easy negatives100000.990.007540.02242
Hard negatives1000.424.7398535.99188
Positive examples200.60.408660.81039

Easy negatives contribute 0.030% of this population’s loss.

Separately, the paper initializes rare foreground at π=0.01 using bias log(π/(1−π)) = -4.59512. Initialization and focal weighting solve different parts of the training setup.

Loss sums and summed absolute per-logit gradients illustrate weighting. They are not parameter-gradient norms, learning curves or evidence of detector AP. Alpha is fixed to 0.25.

With gamma zero, the easy-negative group can dominate because its count is large. Increasing gamma discounts that group's high pₜ much more strongly than the hard examples. Changing the number of easy negatives can restore a large contribution even under focal loss. This experiment illustrates an arithmetic mechanism; it does not reconstruct the paper's observed anchor distribution.

The table also sums absolute derivatives with respect to the individual logits. That is useful for comparing how the scalar objective changes its local emphasis. It is not the norm of the parameter gradient: network parameters are shared, Jacobians differ across examples, and signed gradients can reinforce or cancel. Reporting the scalar sum as measured optimization dynamics would overstate what the calculation establishes.

Difficulty is model-dependent. An example can begin hard and become easy during training, which changes its weight without changing its class label. A mislabeled example may remain hard and therefore receive persistent emphasis. Increasing gamma is consequently not a universal cure for noisy annotations or poor anchor matching.

Initialize the background-heavy problem deliberately

The paper also changes the initial classification bias. A sigmoid initialized near 0.5 assumes equal positive and negative probability before learning. That can create a large initial loss when almost all dense decisions are negative. The reported recipe starts with a rare foreground prior π=0.01 and chooses:

b=\log\frac{\pi}{1-\pi}\approx-4.59512

This changes the initial predicted foreground probability, rather than changing labels or the definition of focal loss. Initialization and confidence-dependent weighting play separate roles. A fair reproduction should retain both when it claims to reproduce the paper's training recipe. Original paper, §3.3

The prior is an optimization setup, not a calibrated estimate guaranteed to remain valid after training. A detector's final scores depend on the learned model, loss, data and postprocessing. Choosing a low prior does not prove that the resulting confidence values equal real-world event probabilities.

The loss was tested inside RetinaNet

RetinaNet combines an FPN backbone with dense classification and box-regression subnetworks. The pyramid supplies representations at multiple spatial scales. Classification uses independent sigmoid decisions, while a separate regression objective learns box adjustments. Focal loss modifies the classification objective; it does not replace the backbone, matching rules or localization loss. Original paper, §4

The paper's controlled ablations train on COCO trainval35k and evaluate on minival, using a default ResNet-50-FPN and 600-pixel image scale. Alpha-balanced cross entropy reaches 31.1 box AP in Table 1(a); gamma two with alpha 0.25 reaches 34.0 AP in Table 1(b), a reported 2.9-point gain. Alpha was selected for each gamma in the comparison, so it is not a pure single-variable gamma sweep. Original paper, Table 1 and §5

The gamma-five configuration obtains 32.2 AP in that table. A stronger focusing effect is therefore not monotonically better on the reported detector. This is a source-specific warning against assuming that more emphasis on hard examples must improve the metric.

The headline 39.1 test-dev AP belongs to RetinaNet-101-800 trained with scale jitter and a schedule 1.5 times longer than its Table 1(e) counterpart. It should not be attributed to the small default ablation setting or to the loss in isolation. These are original reported box-detection results, not reruns here. Original paper, Table 2

The evidence supports the combined detector recipe and the controlled loss comparisons under those settings. It does not establish that focal loss always beats sampling, improves every rare category, or preserves score calibration. A new application should evaluate the metrics it actually needs and inspect which examples the objective emphasizes.

Original objective beside executable checks

The source rows connect the original equation and initialization paragraph to code that evaluates stable loss, its complete derivative and supplied population totals. PDF crops record version, hashes and geometry. The gradient row labels its derivation explicitly, keeping source evidence separate from authored mathematics.

1. Separate class balancing from confidence focusing

Original paper

Equations 4–5: focal loss and alpha balancing

Equation 5 combines the class weight and the confidence-dependent focusing factor.

Math and tensor shapes

\begin{aligned}q_t&=1-p_t\\ \operatorname{FL}&=-\alpha_tq_t^\gamma\log p_t\end{aligned}

For a positive label alpha_t=alpha; for a negative label alpha_t=1−alpha. Gamma zero retains alpha-weighted BCE when alpha is present. Set alpha=None to remove class weighting. Stable log-sigmoid and expm1 avoid arbitrary clipping.

Python implementation

focalloss.py · loss
python
def focal_loss(logit, target, gamma=2.0, alpha=0.25):
    """Stable alpha_t * (1-p_t)^gamma * BCE, with no probability clipping."""
    _, log_pt, _, q, alpha_t = terms(logit, target, gamma, alpha)
    return -alpha_t*q**gamma*log_pt

2. Differentiate both factors of the loss

Original paper

Section 3.2: focusing behavior

The paper defines a probability-dependent modulation; this derivative is calculated here from that definition.

Math and tensor shapes

\begin{aligned}q_t&=1-p_t\\ u_t&=\gamma p_t\log p_t-q_t\\ \frac{\partial L}{\partial z}&=\alpha_t s q_t^\gamma u_t\end{aligned}

Here s=2y−1 for binary targets y in {0,1}. The gamma p_t log p_t term comes from differentiating the focusing factor. Detaching the factor changes the training objective. The implementation handles gamma zero explicitly and tests both labels and fractional gamma.

Python implementation

focalloss.py · gradient
python
def focal_gradient(logit, target, gamma=2.0, alpha=0.25):
    """Differentiate the focusing factor too; do not detach its gradient."""
    sign, log_pt, pt, q, alpha_t = terms(logit, target, gamma, alpha)
    if gamma == 0:
        return -alpha_t*sign*q
    return alpha_t*sign*q**gamma*(gamma*pt*log_pt-q)

3. Sum supplied examples and initialize a rare-class prior

Original paper

Section 3.3: foreground prior initialization

Section 3.3 sets the initial sigmoid probability to a small foreground prior.

Math and tensor shapes

\begin{aligned}L_{\rm group}&=N\,L(z,y)\\ b&=\log\frac{\pi}{1-\pi}\end{aligned}

The population arithmetic uses invented example counts, not the paper’s measured anchor statistics. Summed absolute logit gradients are explanatory values. The prior bias is a separate initialization decision; pi=0.01 gives approximately −4.595.

Python implementation

focalloss.py · population
python
def population_loss(groups, gamma=2.0, alpha=0.25):
    """Count-weighted loss and absolute gradient; no learned data distribution."""
    output = []
    for label, count, pt, target in groups:
        if not isinstance(count, int) or count < 0 or not 0 < pt < 1:
            raise ValueError("Invalid count or probability")
        logit = (2*target-1)*math.log(pt/(1-pt))
        per = focal_loss(logit, target, gamma, alpha)
        output.append({"label": label, "count": count, "per": per,
                       "total": count*per,
                       "gradient": count*abs(focal_gradient(logit, target, gamma, alpha))})
    return output


def prior_bias(prior=0.01):
    """Set a sigmoid bias to a supplied rare-foreground prior."""
    if not 0 < prior < 1:
        raise ValueError("Prior must be strictly between zero and one")
    return math.log(prior/(1-prior))

Original PDF excerpts: Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár. Source 1708.02002v2; exact PDF/image hashes and crop rectangles are recorded.

Run python examples/focalloss.py --json, or download the reference. The README and requirements describe the standard-library environment and numerical assumptions. The program's assertions check gamma-zero limits, class symmetry, finite-difference derivatives, rare-prior initialization and extreme logits.

The practical reading habit is to ask three questions independently: which class multiplier is used, how difficulty changes the scalar loss, and how differentiation changes the update. Keeping those questions separate makes it easier to debug a dense detector and to interpret a loss ablation without claiming that a curve alone explains a benchmark.

If you found this paper review helpful, consider sharing it with others.

Mastodon