🎨 GAN Core Philosophy
🎯 The Adversarial Framework
Generative Adversarial Networks (GANs), introduced by Ian Goodfellow in 2014, are a class of deep learning models that generate new data from the same distribution as the training data. GANs consist of two competing networks: a Generator and a Discriminator.
The Adversarial Game
Generator vs Discriminator — The Generator creates fake data to fool the Discriminator, while the Discriminator tries to distinguish real from fake. This adversarial process drives both networks to improve, ultimately producing high-quality synthetic data.
import tensorflow as tf
from tensorflow.keras import layers
# Generator: latent vector → image
def make_generator():
model = tf.keras.Sequential([
layers.Dense(7*7*256, use_bias=False, input_shape=(100,)),
layers.BatchNormalization(),
layers.ReLU(),
layers.Reshape((7, 7, 256)),
layers.Conv2DTranspose(128, (5,5), strides=(1,1), padding='same', use_bias=False),
layers.BatchNormalization(),
layers.ReLU(),
layers.Conv2DTranspose(64, (5,5), strides=(2,2), padding='same', use_bias=False),
layers.BatchNormalization(),
layers.ReLU(),
layers.Conv2DTranspose(1, (5,5), strides=(2,2), padding='same', use_bias=False, activation='tanh')
])
return model
# Discriminator: image → real/fake
def make_discriminator():
model = tf.keras.Sequential([
layers.Conv2D(64, (5,5), strides=(2,2), padding='same', input_shape=(28,28,1)),
layers.LeakyReLU(0.2),
layers.Dropout(0.3),
layers.Conv2D(128, (5,5), strides=(2,2), padding='same'),
layers.LeakyReLU(0.2),
layers.Dropout(0.3),
layers.Flatten(),
layers.Dense(1, activation='sigmoid')
])
return model🎭 Generator Network
🎯 From Latent to Data
Generator takes random noise (latent vector) and transforms it into realistic data samples. Learns to capture the true data distribution.
Latent Space
The latent space (typically 100-512 dimensions) captures meaningful semantic features. Moving in this space corresponds to smooth changes in generated outputs. The generator learns a mapping from this latent space to the data space.
# Generator architecture (DCGAN style)
def build_generator(z_dim=100):
model = tf.keras.Sequential([
# Project and reshape
layers.Dense(8*8*256, input_dim=z_dim),
layers.Reshape((8, 8, 256)),
layers.BatchNormalization(),
layers.ReLU(),
# Upsampling blocks
layers.Conv2DTranspose(128, (4,4), strides=(2,2), padding='same'),
layers.BatchNormalization(),
layers.ReLU(),
layers.Conv2DTranspose(64, (4,4), strides=(2,2), padding='same'),
layers.BatchNormalization(),
layers.ReLU(),
layers.Conv2DTranspose(3, (4,4), strides=(2,2), padding='same', activation='tanh')
])
return model🕵️ Discriminator Network
🎯 Real vs Fake Classifier
Discriminator is a binary classifier that outputs the probability that a given sample is real (from training data) rather than fake (generated).
Decision Boundary
The discriminator learns to distinguish subtle patterns. As the generator improves, the discriminator must become more sophisticated.
def build_discriminator(img_shape=(64,64,3)):
model = tf.keras.Sequential([
layers.Conv2D(64, (4,4), strides=(2,2), padding='same', input_shape=img_shape),
layers.LeakyReLU(0.2),
layers.Conv2D(128, (4,4), strides=(2,2), padding='same'),
layers.LeakyReLU(0.2),
layers.Conv2D(256, (4,4), strides=(2,2), padding='same'),
layers.LeakyReLU(0.2),
layers.Flatten(),
layers.Dropout(0.3),
layers.Dense(1, activation='sigmoid')
])
return model⚔️ Adversarial Training Dynamics
🎯 The Minimax Game
Min_G Max_D V(D,G) = E[log(D(x))] + E[log(1 - D(G(z)))]. This is a minimax optimization problem.
Training Alternation
Train discriminator on real and fake samples, then train generator to fool the discriminator. This alternation continues until Nash equilibrium is reached.
# Training loop skeleton
for epoch in range(epochs):
# Train discriminator
real_loss = discriminator.train_on_batch(real_images, ones)
fake_images = generator.predict(noise)
fake_loss = discriminator.train_on_batch(fake_images, zeros)
# Train generator
noise = np.random.normal(0, 1, (batch_size, z_dim))
generator_loss = gan.train_on_batch(noise, ones)📉 GAN Loss Functions
🎯 Minimax vs Wasserstein
Original: binary cross-entropy. WGAN: Wasserstein loss (Earth Mover's distance) with gradient penalty.
Loss Evolution
Wasserstein loss provides smoother gradients and better training stability. It uses a critic (instead of discriminator) that outputs a score (not probability).
# WGAN-GP Loss
def wasserstein_loss(y_true, y_pred):
return -tf.reduce_mean(y_true * y_pred)
# Gradient penalty (WGAN-GP)
def gradient_penalty(discriminator, real, fake):
alpha = tf.random.uniform([real.shape[0], 1, 1, 1], 0.0, 1.0)
interpolated = alpha * real + (1 - alpha) * fake
with tf.GradientTape() as tape:
tape.watch(interpolated)
pred = discriminator(interpolated)
grads = tape.gradient(pred, interpolated)
return tf.reduce_mean(tf.square(tf.norm(grads, axis=1) - 1.0))🏛️ DCGAN Architecture
🎯 Deep Convolutional GAN
DCGAN introduced architectural guidelines for stable training: strided convolutions, batch normalization, ReLU in generator, LeakyReLU in discriminator.
Key Design Principles
No pooling layers (use strided convs). BatchNorm everywhere except final layer. Tanh for generator output. LeakyReLU in discriminator.
# DCGAN design principles in action
# Generator: transposed convs with batch norm and ReLU
# Discriminator: convs with batch norm and LeakyReLU
# Both use stride-2 for down/up sampling
# No fully connected layers (except input projection)🌀 Mode Collapse & Training Instability
🎯 The Biggest Challenge
Mode collapse: generator produces limited variety (only a few outputs). Training instability: discriminator becomes too strong.
Mitigation Strategies
Use Wasserstein loss, label smoothing, mini-batch discrimination, unrolled GANs, or spectral normalization.
# Label smoothing
smooth_real = 0.9 # instead of 1.0
smooth_fake = 0.1 # instead of 0.0
# Spectral normalization (in Keras)
from tensorflow.keras.layers import Conv2D
conv = Conv2D(64, 3, kernel_initializer='spectral_normalization')🎯 Conditional GANs (cGAN)
🎯 Controlled Generation
Conditional GANs incorporate additional information (class labels, attributes) to control the generated output.
Conditioning Mechanism
Concatenate condition vector to both generator input (latent + condition) and discriminator input (image + condition). Enables targeted generation.
# Conditional GAN generator
def build_cgan_generator(z_dim=100, num_classes=10):
label_input = layers.Input(shape=(num_classes,))
z_input = layers.Input(shape=(z_dim,))
concat = layers.Concatenate()([z_input, label_input])
# ... rest of generator
return tf.keras.Model([z_input, label_input], output)🖼️ Pix2Pix & Image-to-Image Translation
🎯 Paired Image Translation
Pix2Pix learns mapping from input image to output image using paired data (e.g., edges → photo, maps → satellite images).
U-Net Generator + PatchGAN
U-Net architecture for generator (skip connections). PatchGAN discriminator classifies patches (70x70) to encourage local realism.
# Pix2Pix uses L1 loss + GAN loss
def pix2pix_loss(disc_loss, gen_loss, lambda_l1=100):
return gen_loss + lambda_l1 * tf.reduce_mean(tf.abs(real - generated))🔄 CycleGAN & Unpaired Translation
🎯 Unpaired Image Translation
CycleGAN learns mappings between two domains without paired examples, using cycle consistency loss to preserve content.
Cycle Consistency
F(G(x)) ≈ x (forward cycle) and G(F(y)) ≈ y (backward cycle). This preserves the underlying content during translation.
# CycleGAN has 2 generators and 2 discriminators
# Cycle consistency loss
def cycle_loss(real, reconstructed, lambda_cycle=10):
return lambda_cycle * tf.reduce_mean(tf.abs(real - reconstructed))✨ StyleGAN & Style Transfer
🎯 High-Quality Generation
StyleGAN introduced style-based generation, enabling control over different levels of features (coarse, medium, fine).
Style Modulation
AdaIN (Adaptive Instance Normalization) injects style information at different scales. This allows mixing and manipulation of styles.
# StyleGAN concepts
# Mapping network: latent → style vector
# Synthesis network: style vectors control each layer
# Noise injection adds stochastic variation📊 Evaluation Metrics
🎯 FID, Inception Score, Precision/Recall
FID (Fréchet Inception Distance) measures similarity between real and generated distributions. Lower FID = better quality.
Metric Comparison
FID is the most widely used metric. Inception Score measures quality and diversity. Precision/Recall capture mode coverage.
# Calculate FID using tensorflow
import tensorflow_hub as hub
def calculate_fid(real_images, generated_images):
inception = hub.load('https://tfhub.dev/google/inception_v3/feature_vector/4')
# Extract features and compute Fréchet distance
# Implementation requires mean and covariance comparison🚀 Applications & Use Cases
🎯 Real-World Impact
Image generation, super-resolution, image editing, style transfer, data augmentation, medical imaging, video generation.
Industry Adoption
GANs are used in fashion, gaming, art, pharmaceuticals (molecule generation), and synthetic data for privacy-preserving AI.
# Applications examples
# Super-resolution: SRGAN
# Text-to-image: StackGAN, DALL-E
# Face generation: StyleGAN
# Video generation: MoCoGAN
# 3D generation: 3D-GAN📦 Implementation & Deployment
🎯 Training & Serving GANs
Implement custom training loops, use callbacks for monitoring, deploy with TensorFlow Serving or ONNX.
Production Considerations
Monitor training stability. Use checkpoints. For deployment, separate generator from discriminator. Consider model compression for real-time applications.
# Complete GAN training loop with custom training
@tf.function
def train_step(real_images):
noise = tf.random.normal([BATCH_SIZE, Z_DIM])
with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:
generated_images = generator(noise, training=True)
real_output = discriminator(real_images, training=True)
fake_output = discriminator(generated_images, training=True)
gen_loss = generator_loss(fake_output)
disc_loss = discriminator_loss(real_output, fake_output)
# Apply gradients
gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)
gradients_of_discriminator = disc_tape.gradient(disc_loss, discriminator.trainable_variables)
generator_optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))
discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator, discriminator.trainable_variables))
return gen_loss, disc_loss