🧠 CNN Foundations & Biological Inspiration
🎯 The Vision Revolution
Convolutional Neural Networks (CNNs) were inspired by the visual cortex (Hubel & Wiesel, 1959). They use learnable filters to automatically detect features like edges, textures, and objects. CNNs exploit spatial structure with local connectivity and parameter sharing, making them extremely efficient for image tasks.
Deep Explanation
Key properties: Local connectivity (neurons respond to local regions), parameter sharing (same filter across entire image), and spatial hierarchy (low-level → high-level features). CNNs reduce the number of parameters dramatically compared to fully connected networks, enabling deep architectures.
Why it matters: CNNs power modern computer vision: image classification, object detection, segmentation, medical imaging, autonomous vehicles, and more.
import torch
import torch.nn as nn
# Simple CNN building block
class SimpleCNN(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3, padding=1)
self.relu = nn.ReLU()
self.pool = nn.MaxPool2d(kernel_size=2, stride=2)
def forward(self, x):
x = self.pool(self.relu(self.conv1(x)))
return x
model = SimpleCNN()
print(model)🔲 Convolution Operation
🎯 Sliding Window & Cross-Correlation
Convolution applies a filter (kernel) to the input, computing element-wise multiplication and summing the result. The filter slides across the input, producing a feature map.
Stride, Padding & Dilation
Stride controls the step size; padding preserves spatial dimensions; dilation expands the receptive field. Output size: ((H - K + 2P) / S) + 1.
import torch.nn.functional as F
# Manual convolution
input = torch.randn(1, 1, 5, 5) # (batch, channels, height, width)
kernel = torch.randn(1, 1, 3, 3)
output = F.conv2d(input, kernel, stride=1, padding=1)
print(f"Output shape: {output.shape}")🎯 Filters, Kernels & Feature Maps
🎯 Learnable Feature Extractors
Each filter learns to detect a specific pattern (edge, texture, blob). Multiple filters produce multiple feature maps, capturing different aspects of the input.
Interpretation
Early layers detect simple patterns (edges, colors). Middle layers detect textures and parts. Later layers detect object parts and complete objects.
# Visualizing filters (concept)
# In practice: model.conv1.weight.data shows learned filters
import matplotlib.pyplot as plt
# filters = model.conv1.weight.detach().cpu().numpy()
# plt.imshow(filters[0, 0], cmap='gray')📉 Pooling Layers (Max, Average)
🎯 Spatial Downsampling
Pooling reduces spatial dimensions, increases receptive field, and provides translation invariance. Max pooling selects the maximum value in each window; average pooling takes the mean.
Why Pooling Works
It reduces computational load, controls overfitting, and makes features more robust to small translations. Global average pooling is used at the end of many modern architectures.
pool = nn.MaxPool2d(kernel_size=2, stride=2)
avg_pool = nn.AvgPool2d(kernel_size=2, stride=2)
x = torch.randn(1, 3, 32, 32)
print(f"Max pool output: {pool(x).shape}")
print(f"Avg pool output: {avg_pool(x).shape}")🏗️ Classic Architectures (LeNet, AlexNet, VGG)
🎯 The Pioneers
LeNet-5 (1998) — digit recognition. AlexNet (2012) — deep CNN with ReLU, Dropout, and GPU training. VGG (2014) — very deep networks with 3x3 convolutions.
Evolution
AlexNet proved that deep CNNs can win ImageNet. VGG showed that depth and small filters are key. These architectures established the principles used in modern networks.
# VGG-like block
class VGGBlock(nn.Module):
def __init__(self, in_channels, out_channels):
super().__init__()
self.conv1 = nn.Conv2d(in_channels, out_channels, 3, padding=1)
self.conv2 = nn.Conv2d(out_channels, out_channels, 3, padding=1)
self.pool = nn.MaxPool2d(2, 2)
def forward(self, x):
x = F.relu(self.conv1(x))
x = F.relu(self.conv2(x))
return self.pool(x)🔄 ResNets & Skip Connections
🎯 Solving the Degradation Problem
ResNets (He et al., 2015) introduce skip connections (shortcuts) that add the input to the output of stacked layers, allowing gradients to flow directly.
Identity Mapping
Skip connections enable training of networks with 100+ layers by mitigating vanishing gradients. The residual mapping F(x) + x is easier to learn than the original mapping.
class ResidualBlock(nn.Module):
def __init__(self, channels):
super().__init__()
self.conv1 = nn.Conv2d(channels, channels, 3, padding=1)
self.conv2 = nn.Conv2d(channels, channels, 3, padding=1)
def forward(self, x):
residual = x
out = F.relu(self.conv1(x))
out = self.conv2(out)
out += residual # Skip connection
return F.relu(out)🌐 Inception & 1x1 Convolutions
🎯 Multi-Scale Feature Extraction
Inception (GoogLeNet) uses parallel convolutions of different kernel sizes (1x1, 3x3, 5x5) and pooling, concatenating the outputs. 1x1 convolutions reduce dimensionality.
Efficiency
1x1 convolutions act as channel-wise fully connected layers, reducing the number of parameters while preserving spatial structure.
class InceptionBlock(nn.Module):
def __init__(self, in_channels, out_1x1, out_3x3, out_5x5, out_pool):
super().__init__()
self.conv1 = nn.Conv2d(in_channels, out_1x1, 1)
self.conv3 = nn.Conv2d(in_channels, out_3x3, 3, padding=1)
self.conv5 = nn.Conv2d(in_channels, out_5x5, 5, padding=2)
self.pool = nn.MaxPool2d(3, stride=1, padding=1)
self.pool_conv = nn.Conv2d(in_channels, out_pool, 1)
def forward(self, x):
return torch.cat([
self.conv1(x),
self.conv3(x),
self.conv5(x),
self.pool_conv(self.pool(x))
], dim=1)🔄 Transposed Convolutions
🎯 Upsampling & Reconstruction
Transposed convolutions (deconvolutions) perform the inverse of convolution: upsample feature maps. Used in autoencoders, GANs, and segmentation networks.
Learnable Upsampling
Unlike fixed interpolation, transposed convolutions learn the upsampling parameters. They can create checkerboard artifacts, so they require careful use.
transposed_conv = nn.ConvTranspose2d(32, 16, kernel_size=4, stride=2, padding=1)
x = torch.randn(1, 32, 16, 16)
out = transposed_conv(x)
print(f"Upsampled: {out.shape}") # -> (1, 16, 32, 32)👁️ Attention & Self-Attention in CNNs
🎯 Focusing on Important Regions
Attention mechanisms allow networks to focus on relevant parts of the input. Self-attention (from Transformers) has been integrated into CNNs for improved performance.
Non-Local Networks
Non-local blocks compute pairwise relationships between all positions, capturing long-range dependencies that standard convolutions cannot.
# Simplified self-attention for images
class SelfAttention(nn.Module):
def __init__(self, channels):
super().__init__()
self.query = nn.Conv2d(channels, channels//8, 1)
self.key = nn.Conv2d(channels, channels//8, 1)
self.value = nn.Conv2d(channels, channels, 1)
def forward(self, x):
q = self.query(x).view(x.size(0), -1, x.size(2)*x.size(3))
k = self.key(x).view(x.size(0), -1, x.size(2)*x.size(3))
v = self.value(x).view(x.size(0), -1, x.size(2)*x.size(3))
attn = torch.softmax(q.transpose(1,2) @ k, dim=-1)
out = (v @ attn.transpose(1,2)).view(x.shape)
return out + x🎯 Object Detection (R-CNN, YOLO)
🎯 Finding & Classifying Objects
R-CNN family (region-based) and YOLO (single-shot) are the two major paradigms. R-CNN is accurate but slow; YOLO is fast and real-time.
Anchor Boxes & NMS
Anchor boxes are predefined bounding boxes. Non-Maximum Suppression (NMS) removes duplicate detections.
# YOLO-like output: (batch, boxes, 5 + classes)
# Each box: (x, y, w, h, confidence, class_probs)
# Non-Maximum Suppression
def nms(boxes, scores, iou_threshold=0.5):
# Implementation details omitted for brevity
pass🧩 Semantic Segmentation (FCN, UNet)
🎯 Pixel-Level Classification
Fully Convolutional Networks (FCN) replace dense layers with convolutions. UNet uses a symmetric encoder-decoder with skip connections for medical images.
Encoder-Decoder
The encoder extracts features, the decoder upsamples to original resolution. Skip connections preserve spatial details.
# UNet-like skip connection
class UNetBlock(nn.Module):
def __init__(self, in_channels, out_channels):
super().__init__()
self.conv = nn.Conv2d(in_channels, out_channels, 3, padding=1)
def forward(self, x, skip):
x = F.relu(self.conv(x))
return torch.cat([x, skip], dim=1) # Skip connection🔄 Transfer Learning & Fine-Tuning
🎯 Leveraging Pre-trained Models
Pre-trained on ImageNet (1.2M images, 1000 classes). Fine-tuning: replace the classification head and retrain on a smaller dataset.
Why Transfer Learning Works
Early layers learn generic features (edges, textures). These transfer to most visual tasks. Only the later task-specific layers need retraining.
import torchvision.models as models
# Load pre-trained ResNet
model = models.resnet50(pretrained=True)
# Freeze all layers
for param in model.parameters():
param.requires_grad = False
# Replace the final layer
model.fc = nn.Linear(2048, 10) # 10 classes
# Fine-tune only the new layer⚙️ Training: Data Augmentation, Regularization
🎯 Preventing Overfitting
Data augmentation (rotation, flip, crop, color jitter) increases dataset diversity. Regularization: Dropout, weight decay, batch normalization.
Batch Normalization
Normalizes activations, speeds up training, and acts as a regularizer. Essential for deep CNNs.
from torchvision import transforms
transform = transforms.Compose([
transforms.RandomHorizontalFlip(),
transforms.RandomRotation(10),
transforms.ColorJitter(brightness=0.2, contrast=0.2),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])🚀 Deployment & Production
🎯 From Model to Service
Export to ONNX, TensorRT, or PyTorch JIT for optimized inference. Deploy on cloud (AWS, GCP), edge devices (Jetson, mobile), or browsers (TensorFlow.js).
Optimization Techniques
Quantization (FP16, INT8), pruning, and knowledge distillation reduce model size and latency.
# Export to ONNX
torch.onnx.export(model, dummy_input, "model.onnx",
input_names=['input'], output_names=['output'])
# For serving: FastAPI + PyTorch
# model.eval(); with torch.no_grad(): out = model(input)