Backpropagation Through Time (BPTT)Open in Colab

Backpropagation Through Time (BPTT)

BPTT is ordinary backpropagation applied to an unrolled RNN. The key challenge: the gradient of a loss at step TT with respect to a hidden state at step kk requires a product of T−kT-k Jacobians — one per intermediate step.

The chain rule through time

To update weights using the loss at step TT, we need ∂LT/∂hk\partial \mathcal{L}_T / \partial \mathbf{h}_k for all earlier states k<Tk < T. The only path from hk\mathbf{h}_k to LT\mathcal{L}_T runs forward through every hidden state in between. The chain rule threads through hk+1,…,hT\mathbf{h}_{k+1}, \ldots, \mathbf{h}_T:

∂LT∂hk=∂LT∂hT ∂hT∂hT−1⋯∂hk+1∂hk=∂LT∂hT∏t=k+1T∂ht∂ht−1\frac{\partial \mathcal{L}_T}{\partial \mathbf{h}_k} = \frac{\partial \mathcal{L}_T}{\partial \mathbf{h}_T}\, \frac{\partial \mathbf{h}_T}{\partial \mathbf{h}_{T-1}}\cdots \frac{\partial \mathbf{h}_{k+1}}{\partial \mathbf{h}_k} = \frac{\partial \mathcal{L}_T}{\partial \mathbf{h}_T} \prod_{t=k+1}^{T} \frac{\partial \mathbf{h}_t}{\partial \mathbf{h}_{t-1}}

The gradient over a gap of T−kT-k steps is a product of T−kT-k Jacobian matrices. Everything hinges on what one of those Jacobians looks like.

Deriving one step's Jacobian

The recurrence is ht=tanh⁡(at)\mathbf{h}_t = \tanh(\mathbf{a}_t), where the pre-activation is:

at=Whhht−1+Wxhxt+b\mathbf{a}_t = W_{hh}\mathbf{h}_{t-1} + W_{xh}\mathbf{x}_t + \mathbf{b}

Differentiate component-by-component. For output unit ii and input unit jj:

∂ht,i∂ht−1,j=tanh⁡′(at,i)⏟outer: activation⋅∂at,i∂ht−1,j⏟=(Whh)ij=tanh⁡′(at,i) (Whh)ij\frac{\partial h_{t,i}}{\partial h_{t-1,j}} = \underbrace{\tanh'(a_{t,i})}_{\text{outer: activation}} \cdot\underbrace{\frac{\partial a_{t,i}}{\partial h_{t-1,j}}}_{= (W_{hh})_{ij}} = \tanh'(a_{t,i})\,(W_{hh})_{ij}

The tanh⁡′(at,i)\tanh'(a_{t,i}) factor depends only on row ii, so collecting all i,ji, j entries into a matrix puts those factors on a diagonal:

  ∂ht∂ht−1=diag ⁣(tanh⁡′(at)) Whh  \boxed{\;\frac{\partial \mathbf{h}_t}{\partial \mathbf{h}_{t-1}} = \mathrm{diag}\!\big(\tanh'(\mathbf{a}_t)\big)\,W_{hh}\;}

In backprop the gradient propagates backward using the transpose:

dht−1=Whh⊤ diag(tanh⁡′(at)) dht\mathbf{d}\mathbf{h}_{t-1} = W_{hh}^{\top}\,\mathrm{diag}(\tanh'(\mathbf{a}_t))\,\mathbf{d}\mathbf{h}_t

Each step back multiplies by one copy of WhhW_{hh} and one diagonal of tanh⁡′\tanh' values.

Watch it run

The backward recursion on a 2-unit tanh RNN unrolled over six steps, with the gradient norm reported at every hop:

BPTT — one Jacobian per step, and what their product does

1/11

A 2-unit tanh RNN unrolled over 6 steps: h_t = tanh(W_hh·h_{t−1} + W_xh·x_t + b). BPTT is nothing more than backpropagation on this unrolled graph — but the graph has a special shape, because the *same* W_hh appears at every step, and that repetition is where all the trouble comes from.

python
1def bptt(xs, h0, W_hh, W_xh, b):
2 hs, as_ = [h0], []
3 for x in xs: # forward
4 a = W_hh @ hs[-1] + W_xh @ x + b
5 as_.append(a)
6 hs.append(tanh(a))
7 dh = dloss_dh(hs[-1]) # seed at t = T
8 norms = [norm(dh)]
9 for t in reversed(range(len(xs))):
10 # one step back = one W_hh and one tanh'
11 d = 1 - tanh(as_[t]) ** 2 # tanh'(a_t)
12 dh = W_hh.T @ (d * dh)
13 norms.append(norm(dh))
14 return norms
state
W_hh (spectral radius 0.6)
01
→00.5000.100
→10.1000.500
hidden states h_t
unit 0unit 1
h₀0.100-0.100
h1··
h2··
h3··
h4··
h5··
h6··

The backward recursion on a 2-unit tanh RNN unrolled over six steps. Each step back multiplies by one W_hhᵀ and one diagonal of tanh′ values — watch the gradient norm shrink by a fixed factor every single time, because the same matrix is applied at every step. The final step sweeps the spectral radius of W_hh and plots the result on a log axis: geometric decay below 1, geometric growth above it, and a knife-edge in between that no initialization can hold.

What to notice as you step:

  • Every backward step is the same two multiplications. One Whh⊤W_{hh}^\top and one diagonal of tanh⁡′\tanh' values, and because it is the same WhhW_{hh} each time, the shrink factor is roughly constant, which makes the decay geometric rather than incidental.
  • tanh⁡′\tanh' can only make it worse. It is at most 1, and strictly below 1 whenever a unit is saturated at all, so it never rescues a small WhhW_{hh}.
  • After six steps the gradient is 0.7% of what it started as. The loss at step 6 has essentially nothing to say about h0\mathbf{h}_0, and that gradient is the learning signal for the earliest timesteps.
  • The final step sweeps the spectral radius on a log axis. Below 1 the curve is a straight line downward, above 1 a straight line upward, and the usable region is a knife-edge. Clipping fixes the exploding side cheaply; nothing fixes the vanishing side, which is why LSTMs introduced an additive cell path with no repeated matrix on it.

The full product over a gap

Substituting the Jacobian into the product, the gap-(T−k)(T-k) gradient is:

∂LT∂hk=∂LT∂hT∏t=k+1Tdiag ⁣(tanh⁡′(at)) Whh\frac{\partial \mathcal{L}_T}{\partial \mathbf{h}_k} = \frac{\partial \mathcal{L}_T}{\partial \mathbf{h}_T} \prod_{t=k+1}^{T} \mathrm{diag}\!\big(\tanh'(\mathbf{a}_t)\big)\,W_{hh}

This product is the source of both vanishing and exploding gradients. See the companion page for the spectral-norm analysis:

Deep dive
Vanishing & Exploding Gradients: Spectral Analysis
Includes runnable notebook

A worked example: a 1-D RNN over three steps

Scalar RNN with Whh=0.8W_{hh} = 0.8, Wxh=0.5W_{xh} = 0.5, sequence x=[1,0,1]\mathbf{x} = [1, 0, 1], h0=0h_0 = 0.

Step 1: forward pass. Compute pre-activations, hidden states, and tanh⁡′\tanh':

ttat=0.8 ht−1+0.5 xta_t = 0.8\,h_{t-1} + 0.5\,x_tht=tanh⁡(at)h_t = \tanh(a_t)tanh⁡′(at)=1−ht2\tanh'(a_t) = 1 - h_t^2
10.8(0)+0.5(1)=0.5000.8(0) + 0.5(1) = 0.5000.4620.4620.7870.787
20.8(0.462)+0.5(0)=0.3700.8(0.462) + 0.5(0) = 0.3700.3540.3540.8750.875
30.8(0.354)+0.5(1)=0.7830.8(0.354) + 0.5(1) = 0.7830.6550.6550.5710.571

Step 2: BPTT from T=3T=3 back to k=1k=1. Starting from ∂L3/∂h3=1\partial \mathcal{L}_3 / \partial h_3 = 1:

∂h3∂h2=tanh⁡′(a3)⋅Whh=0.571×0.8=0.457\frac{\partial h_3}{\partial h_2} = \tanh'(a_3) \cdot W_{hh} = 0.571 \times 0.8 = 0.457 ∂h2∂h1=tanh⁡′(a2)⋅Whh=0.875×0.8=0.700\frac{\partial h_2}{\partial h_1} = \tanh'(a_2) \cdot W_{hh} = 0.875 \times 0.8 = 0.700 ∂L3∂h1=1×0.457×0.700=0.320\frac{\partial \mathcal{L}_3}{\partial h_1} = 1 \times 0.457 \times 0.700 = \mathbf{0.320}

After just 2 steps backward, the gradient is already at 32% of the upstream signal. With Whh<1W_{hh} < 1 and tanh⁡′≤1\tanh' \le 1, it continues shrinking each step further back.

Python
import numpy as np

W_hh, W_xh = 0.8, 0.5
x_seq = [1, 0, 1]
h = [0.0]
a_vals = []

for x_t in x_seq:
    a_t = W_hh * h[-1] + W_xh * x_t
    a_vals.append(a_t)
    h.append(float(np.tanh(a_t)))

tanh_prime = [1 - np.tanh(a)**2 for a in a_vals]
print("h:", [round(v, 3) for v in h])
print("tanh':", [round(v, 3) for v in tanh_prime])

# BPTT from step 3 back to step 1 (0-indexed: steps 2 and 1)
grad = 1.0
for t in reversed(range(1, 3)):   # t = 2, then 1
    step_grad = tanh_prime[t] * W_hh
    grad *= step_grad
    print(f"After back through t={t+1}: per-step={step_grad:.3f}, cumulative grad={grad:.4f}")
# Output:
# After back through t=3: per-step=0.457, cumulative grad=0.4571
# After back through t=2: per-step=0.700, cumulative grad=0.3200

Why weight-sharing accumulates gradients

Because WhhW_{hh} is reused at every step, its gradient is the sum of contributions from all steps:

∂L∂Whh=∑t=1T∂Lt∂Whh\frac{\partial \mathcal{L}}{\partial W_{hh}} = \sum_{t=1}^{T} \frac{\partial \mathcal{L}_t}{\partial W_{hh}}

Each term ∂Lt/∂Whh\partial \mathcal{L}_t / \partial W_{hh} itself requires back-propagating through all earlier steps to reach WhhW_{hh}, which is exactly the product-of-Jacobians computation above, applied at each tt.

BPTT in PyTorch

In practice PyTorch handles BPTT automatically through the unrolled computational graph:

Python
import torch
import torch.nn as nn

rnn = nn.RNN(input_size=1, hidden_size=4, batch_first=True)
x = torch.randn(1, 10, 1)  # (batch, seq_len, input)
h0 = torch.zeros(1, 1, 4)

output, _ = rnn(x, h0)
loss = output.sum()
loss.backward()  # BPTT runs automatically

print("W_hh gradient shape:", rnn.weight_hh_l0.grad.shape)  # (4, 4)

For long sequences, detach the hidden state at each chunk boundary to truncate the graph (Truncated BPTT):

Python
hidden = hidden.detach()  # keep the value, drop the gradient history

BPTT = standard backprop on a deep net

Unrolling an RNN over TT steps produces a TT-layer feedforward network with tied weights. BPTT is just backpropagation on that deep net — the only novelty is that the same WhhW_{hh} appears in every layer, so its gradient is accumulated across layers rather than computed once.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
Backpropagation Through Time (BPT…Automatic Differentiation…Vanishing & Exploding Gra…BPTT & the Vanishing Grad…LSTM & GRUChain Rule and Backpropag…