Skip to main content
Abhik Sarkar's avatar

GAN: Generative Adversarial Networks

How GANs frame generative modeling as a two-player minimax game: a generator turns noise into samples while a discriminator learns to tell real from fake, driving the generator toward the true data distribution.

TL;DR

  • GANs pit two networks against each other: a Generator G maps random noise to synthetic samples, and a Discriminator D tries to tell real data from G’s fakes. They train together until D can no longer tell them apart.
  • The training objective is a minimax game — D maximizes its classification accuracy while G minimizes D’s ability to succeed, forcing G to produce increasingly realistic samples.
  • At the theoretical optimum the generated distribution exactly matches the data distribution and D outputs exactly 1/2 everywhere, meaning it is maximally confused.
  • GANs produced the first photorealistic generated images and launched a decade of generative modeling research, later challenged but not displaced by diffusion models.

The two-player game

The central insight of Goodfellow et al. is to frame generation as a game between two neural networks rather than as a likelihood-maximization problem. The Generator G takes a latent noise vector z ∼ pz(z) and maps it to a synthetic sample G(z). The Discriminator D takes any sample — real or generated — and outputs a scalar probability that the sample came from the real data distribution.

The two networks are trained simultaneously with opposite objectives. D wants to assign high probability to real samples and low probability to generated ones. G wants to produce samples that fool D into assigning high probability. This produces the minimax objective:

minG maxD \; 𝔼x ∼ p_{\text{data}}[log D(x)] \; + \; 𝔼z ∼ pz[log(1 - D(G(z)))]

D maximizes both terms: it wants D(x) \to 1 for real samples (making log D(x) large) and D(G(z)) \to 0 for fakes (making log(1 - D(G(z))) large). G minimizes the second term, pushing D(G(z)) \to 1 so the discriminator is fooled. In practice, early training uses a non-saturating variant — G maximizes log D(G(z)) rather than minimizing log(1 - D(G(z))) — which provides stronger gradients when G is far from the optimum.

Learning the data distribution

The game has a clean theoretical analysis. For a fixed G, the optimal discriminator is:

D^*(x) = p\text{data}(x)p\text{data}(x) + pg(x)

where pg is the distribution induced by G. Substituting D^* back into the objective yields the global minimum at pg = p\text{data}, where the objective value equals -log 4. At this point D^*(x) = 1/2 everywhere — the discriminator is maximally confused because the generated distribution is indistinguishable from the real one.

The proof also shows that the objective is equivalent to minimizing the Jensen–Shannon divergence between p\text{data} and pg:

C(G) = -log 4 + 2 · \text{JSD}(p\text{data} \| pg)

As training progresses, pg gradually shifts toward p\text{data} and D’s decision boundary flattens toward 0.5. This convergence is visible in the distribution plot below.

Mode collapse

GANs are notoriously difficult to train. The most common failure is mode collapse: G discovers that it can fool D by mastering a small subset of the data distribution (one or two modes) and repeating them, rather than covering the full diversity of the real data. D then learns to reject those specific outputs, G shifts to a different mode, and the two networks cycle without ever covering the whole distribution.

Mode collapse is a consequence of the adversarial objective: G only needs to fool D, not to be diverse. There is no explicit penalty for ignoring modes. Later work addressed this through techniques like minibatch discrimination, unrolled GANs, spectral normalization (SN-GAN), and Wasserstein distance training (WGAN), all of which either penalize collapse or use a more stable training signal.

Why it mattered

GANs changed what generative modeling was capable of. Before 2014, likelihood-based models (VAEs, RBMs, autoregressive models) produced blurry or low-resolution samples because they optimized pixel-level reconstruction objectives. GANs optimized a perceptual criterion implicitly — D learns what “real” looks like, and G is forced to match it.

The cascade of follow-on work was immediate. DCGAN (2015) stabilized GAN training with convolutional architectures and batch normalization. pix2pix (2017) extended GANs to conditional image-to-image translation. StyleGAN and StyleGAN2 (2019–2020) produced photorealistic face synthesis indistinguishable from photographs at 1024×1024 resolution. BigGAN (2018) brought class-conditional ImageNet generation to high fidelity.

The GAN framework also reframed generation as adversarial learning — a design pattern that found applications well beyond images, including text generation, domain adaptation, data augmentation, and neural rendering.

The challenge to GANs came from diffusion models. DDPM (2020) and its successors matched or exceeded GAN sample quality while being dramatically more stable to train, with no adversarial dynamics, no mode collapse, and a proper likelihood objective. Today most state-of-the-art image generation systems (Stable Diffusion, DALL-E 3, Imagen) are diffusion-based. But GANs remain the fastest generative architecture at inference time — a single forward pass of G versus hundreds of diffusion denoising steps — making them attractive where latency matters.

  • DDPM — denoising diffusion probabilistic models, the stable-training alternative that later surpassed GANs in image quality
  • Latent Diffusion — diffusion in a compressed latent space, enabling high-resolution generation at practical compute cost
  • VAE — variational autoencoders, an earlier probabilistic generative framework that GANs were benchmarked against
  • Flow Matching — a modern generative approach that learns straight probability-flow paths, avoiding both GAN instability and diffusion’s slow sampling

If you found this paper review helpful, consider sharing it with others.

Mastodon