Pooling and CNN Architectures
After convolution, we need to reduce spatial dimensions and build deeper representations. Pooling does the downsampling, and famous architectures show how to stack layers effectively.
Pooling
Pooling reduces the spatial size of feature maps, decreasing computation and providing translation invariance.
Max pooling
Takes the maximum value in each window:
import torch.nn as nn
# Max pooling: 2x2 window, stride 2 → halves spatial dimensions
pool = nn.MaxPool2d(kernel_size=2, stride=2)
# Input: [1, 64, 28, 28] → Output: [1, 64, 14, 14]
x = torch.randn(1, 64, 28, 28)
print(f"Before: {x.shape}") # [1, 64, 28, 28]
print(f"After: {pool(x).shape}") # [1, 64, 14, 14]
Average pooling
Takes the mean value in each window. Less common in modern architectures but used in the final layer (Global Average Pooling).
Max vs Average pooling
Max pooling preserves the strongest activation — "did this feature appear anywhere in the region?" Average pooling preserves the average activation — "how strong was this feature overall?" Max pooling is more common because it's better at detecting presence of features.
Global Average Pooling (GAP)
Reduces each feature map to a single value by averaging all pixels:
GAP replaces fully connected layers at the end of the network, drastically reducing parameters:
# GAP: [1, 512, 7, 7] → [1, 512]
gap = nn.AdaptiveAvgPool2d(1)
x = torch.randn(1, 512, 7, 7)
print(f"After GAP: {gap(x).shape}") # [1, 512, 1, 1]
Worked example
A feature map, window, stride 2 (the four non-overlapping quadrants):
Top-left quadrant : max , avg . Each quadrant collapses to one number, so .
Output size formula
Pooling uses the same size formula as convolution — for window , stride , padding :
| Operation | Window | Stride | Padding | Output | e.g. |
|---|---|---|---|---|---|
| MaxPool | 2 | 0 | |||
| MaxPool | 2 | 1 | |||
| GAP | — | — | 1 |
So a chain of stride-2 pools halves the map twice (a final with stride 2 gives ).
Watch the pooling window slide across the feature map — each position produces one output cell:
Pooling: sliding window downsampling
A 2×2 pooling window slides over the 6×6 input. Max pooling keeps the largest value (preserving strong activations); average pooling takes the mean (smoother output). Stride 2 halves each spatial dimension.
Famous architectures
LeNet-5 (1998)
The first successful CNN for digit recognition:
Input (32×32) → Conv(6) → Pool → Conv(16) → Pool → FC(120) → FC(84) → FC(10)
AlexNet (2012)
Won ImageNet with a huge margin. Key innovations:
- Used ReLU instead of sigmoid/tanh
- Dropout for regularization
- Data augmentation (flips, crops)
- GPU training
VGGNet (2014)
Showed that depth matters. Uses only filters:
[Conv3-64]×2 → Pool → [Conv3-128]×2 → Pool → [Conv3-256]×3 → Pool → ...
Why only ? Two stacked beat one . Stacking convolutions of size (stride 1) grows the receptive field linearly:
Each layer adds to the reach (the first sees inputs; every layer after extends both edges by ). For : one layer sees , two layers see — the same receptive field as a single . But the parameter counts differ. For input and output channels:
- one :
- two :
That's 28% fewer parameters for the same receptive field (e.g. : vs ) — and an extra ReLU in between for more non-linearity. This is the whole VGG design philosophy.
ResNet (2015)
Introduced skip connections to train very deep networks (152+ layers):
The skip connection allows gradients to flow directly through the network, solving the vanishing gradient problem.
Why skip connections work
Without skip connections, gradients must flow through every layer. In a 50-layer network, the gradient is multiplied by 50 weight matrices. It can vanish or explode. Skip connections create "gradient highways" that let errors flow directly to early layers.
class ResidualBlock(nn.Module):
def __init__(self, channels):
super().__init__()
self.conv1 = nn.Conv2d(channels, channels, 3, padding=1)
self.bn1 = nn.BatchNorm2d(channels)
self.conv2 = nn.Conv2d(channels, channels, 3, padding=1)
self.bn2 = nn.BatchNorm2d(channels)
def forward(self, x):
residual = x # save input
out = F.relu(self.bn1(self.conv1(x)))
out = self.bn2(self.conv2(out))
out += residual # skip connection!
return F.relu(out)
The evolution at a glance
| Architecture | Year | Key Innovation | Params |
|---|---|---|---|
| LeNet | 1998 | First practical CNN | 60K |
| AlexNet | 2012 | ReLU, Dropout, GPU | 60M |
| VGG-16 | 2014 | Depth with 3×3 filters | 138M |
| ResNet-50 | 2015 | Skip connections | 25M |
| EfficientNet | 2019 | Compound scaling | 5M-66M |
The mechanics behind the modern rows (1×1 convolutions, bottleneck blocks, depthwise separability, compound scaling) are Modern Architectures.
What the layers learn
Peeking inside a trained CNN reveals a hierarchy:
- Layer 1: Edges, colors, simple textures
- Layer 2: Corners, patterns, textures
- Layer 3: Parts of objects (eyes, wheels, leaves)
- Layer 4-5: Whole objects (faces, cars, animals)
This hierarchical feature learning is what makes CNNs powerful — they automatically discover the right representations.
Practical tip
When using a pretrained CNN, the early layers already detect useful features. You can reuse them for new tasks — the foundation of transfer learning. And the tools that show you these features (saliency maps, Grad-CAM) are in CNN Visualization & Adversarial Attacks.
Exercise
What does max pooling with a 2×2 window and stride 2 do to the spatial dimensions?
Need a hint?
Think about how many non-overlapping windows fit.
Exercise
What problem do skip connections in ResNets solve?
Need a hint?
Think about what happens to gradients in very deep networks.
Exercise
Which architecture introduced the concept of skip connections?
Need a hint?
It enabled training networks with 152+ layers.
Common mistakes
- Using pooling everywhere: Modern architectures prefer strided convolutions (stride=2) over max pooling. Pooling loses spatial information.
- Not using batch normalization: BN stabilizes training and allows higher learning rates. Always use it after conv layers.
- Ignoring the classifier head: GAP + a single FC layer is better than multiple FC layers. Fewer parameters, less overfitting.
Key takeaways
- Pooling shrinks spatial size, adds translation invariance, and cuts compute.
- Architectures evolved LeNet → AlexNet → VGG → ResNet (skip connections).
- Skip connections let gradients bypass layers, enabling 100+ layer nets.
- Global average pooling replaces giant fully-connected heads in modern CNNs.
Related concepts
- The Convolution Operation: the operation these networks are built from
- Transfer Learning: reusing these pretrained architectures
- Gradient Descent: how deep stacks are trained