Backpropagation Through Time (BPTT)
BPTT is ordinary backpropagation applied to an unrolled RNN. The key challenge: the gradient of a loss at step with respect to a hidden state at step requires a product of Jacobians — one per intermediate step.
The chain rule through time
To update weights using the loss at step , we need for all earlier states . The only path from to runs forward through every hidden state in between. The chain rule threads through :
The gradient over a gap of steps is a product of Jacobian matrices. Everything hinges on what one of those Jacobians looks like.
Deriving one step's Jacobian
The recurrence is , where the pre-activation is:
Differentiate component-by-component. For output unit and input unit :
The factor depends only on row , so collecting all entries into a matrix puts those factors on a diagonal:
In backprop the gradient propagates backward using the transpose:
Each step back multiplies by one copy of and one diagonal of values.
Watch it run
The backward recursion on a 2-unit tanh RNN unrolled over six steps, with the gradient norm reported at every hop:
BPTT — one Jacobian per step, and what their product does
A 2-unit tanh RNN unrolled over 6 steps: h_t = tanh(W_hh·h_{t−1} + W_xh·x_t + b). BPTT is nothing more than backpropagation on this unrolled graph — but the graph has a special shape, because the *same* W_hh appears at every step, and that repetition is where all the trouble comes from.
| 0 | 1 | |
|---|---|---|
| →0 | 0.500 | 0.100 |
| →1 | 0.100 | 0.500 |
| unit 0 | unit 1 | |
|---|---|---|
| h₀ | 0.100 | -0.100 |
| h1 | · | · |
| h2 | · | · |
| h3 | · | · |
| h4 | · | · |
| h5 | · | · |
| h6 | · | · |
The backward recursion on a 2-unit tanh RNN unrolled over six steps. Each step back multiplies by one W_hhᵀ and one diagonal of tanh′ values — watch the gradient norm shrink by a fixed factor every single time, because the same matrix is applied at every step. The final step sweeps the spectral radius of W_hh and plots the result on a log axis: geometric decay below 1, geometric growth above it, and a knife-edge in between that no initialization can hold.
What to notice as you step:
- Every backward step is the same two multiplications. One and one diagonal of values, and because it is the same each time, the shrink factor is roughly constant, which makes the decay geometric rather than incidental.
- can only make it worse. It is at most 1, and strictly below 1 whenever a unit is saturated at all, so it never rescues a small .
- After six steps the gradient is 0.7% of what it started as. The loss at step 6 has essentially nothing to say about , and that gradient is the learning signal for the earliest timesteps.
- The final step sweeps the spectral radius on a log axis. Below 1 the curve is a straight line downward, above 1 a straight line upward, and the usable region is a knife-edge. Clipping fixes the exploding side cheaply; nothing fixes the vanishing side, which is why LSTMs introduced an additive cell path with no repeated matrix on it.
The full product over a gap
Substituting the Jacobian into the product, the gap- gradient is:
This product is the source of both vanishing and exploding gradients. See the companion page for the spectral-norm analysis:
A worked example: a 1-D RNN over three steps
Scalar RNN with , , sequence , .
Step 1: forward pass. Compute pre-activations, hidden states, and :
| 1 | |||
| 2 | |||
| 3 |
Step 2: BPTT from back to . Starting from :
After just 2 steps backward, the gradient is already at 32% of the upstream signal. With and , it continues shrinking each step further back.
import numpy as np
W_hh, W_xh = 0.8, 0.5
x_seq = [1, 0, 1]
h = [0.0]
a_vals = []
for x_t in x_seq:
a_t = W_hh * h[-1] + W_xh * x_t
a_vals.append(a_t)
h.append(float(np.tanh(a_t)))
tanh_prime = [1 - np.tanh(a)**2 for a in a_vals]
print("h:", [round(v, 3) for v in h])
print("tanh':", [round(v, 3) for v in tanh_prime])
# BPTT from step 3 back to step 1 (0-indexed: steps 2 and 1)
grad = 1.0
for t in reversed(range(1, 3)): # t = 2, then 1
step_grad = tanh_prime[t] * W_hh
grad *= step_grad
print(f"After back through t={t+1}: per-step={step_grad:.3f}, cumulative grad={grad:.4f}")
# Output:
# After back through t=3: per-step=0.457, cumulative grad=0.4571
# After back through t=2: per-step=0.700, cumulative grad=0.3200
Why weight-sharing accumulates gradients
Because is reused at every step, its gradient is the sum of contributions from all steps:
Each term itself requires back-propagating through all earlier steps to reach , which is exactly the product-of-Jacobians computation above, applied at each .
BPTT in PyTorch
In practice PyTorch handles BPTT automatically through the unrolled computational graph:
import torch
import torch.nn as nn
rnn = nn.RNN(input_size=1, hidden_size=4, batch_first=True)
x = torch.randn(1, 10, 1) # (batch, seq_len, input)
h0 = torch.zeros(1, 1, 4)
output, _ = rnn(x, h0)
loss = output.sum()
loss.backward() # BPTT runs automatically
print("W_hh gradient shape:", rnn.weight_hh_l0.grad.shape) # (4, 4)
For long sequences, detach the hidden state at each chunk boundary to truncate the graph (Truncated BPTT):
hidden = hidden.detach() # keep the value, drop the gradient history
BPTT = standard backprop on a deep net
Unrolling an RNN over steps produces a -layer feedforward network with tied weights. BPTT is just backpropagation on that deep net — the only novelty is that the same appears in every layer, so its gradient is accumulated across layers rather than computed once.
Related concepts
- Vanishing & Exploding Gradients: spectral analysis of the Jacobian product
- LSTM & GRU: gated cells that maintain a near-unit Jacobian over long gaps
- Chain Rule & Backpropagation: the single-step rule that BPTT extends