DDIM SamplingOpen in Colab

DDIM Sampling

Standard DDPM sampling requires walking back through all T=1000T = 1000 noise levels — one network call per step. DDIM (Denoising Diffusion Implicit Models, Song et al., 2021) reformulates the reverse process as a deterministic ODE, which makes step-skipping valid: you can visit 50 of the 1000 timesteps and still land on a high-quality sample.

Why step-skipping is valid

DDPM's reverse process is stochastic — each step adds fresh noise, so the trajectory is a random walk that must be taken one step at a time. DDIM removes the stochasticity: given the model's noise prediction, both the "clean image estimate" and the next point on the trajectory are deterministic functions of the current point. A deterministic trajectory can be integrated with large steps, exactly like solving an ODE with a coarser grid.

The DDIM sampling procedure

Recall the cumulative noise coefficient αˉt=∏s=1t(1−βs)\bar{\alpha}_t = \prod_{s=1}^{t}(1-\beta_s), where βs\beta_s is the noise variance added at step ss. Because βs\beta_s is small, αˉt\bar{\alpha}_t decreases from ≈1\approx 1 (clean) to ≈0\approx 0 (pure noise) as tt increases from 00 to TT.

  1. Choose a subsequence τ=[τ1>τ2>⋯>τS]\tau = [\tau_1 > \tau_2 > \cdots > \tau_S] of timesteps to visit (e.g. every 20th step for S=50S=50).
  2. Start from pure noise xτ1∼N(0,I)\mathbf{x}_{\tau_1} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}).
  3. At each step ss (going from τs\tau_s down to τs+1\tau_{s+1}):
    • Run the network to predict the noise: ϵ^=ϵθ(xτs,τs)\hat{\boldsymbol{\epsilon}} = \boldsymbol{\epsilon}_\theta(\mathbf{x}_{\tau_s}, \tau_s).
    • Estimate the clean image:
x^0=xτs−1−αˉτs ϵ^αˉτs\hat{\mathbf{x}}_0 = \frac{\mathbf{x}_{\tau_s} - \sqrt{1-\bar{\alpha}_{\tau_s}}\,\hat{\boldsymbol{\epsilon}}}{\sqrt{\bar{\alpha}_{\tau_s}}}
  • Step to the next timestep by re-noising the estimate to the target noise level:
xτs+1=αˉτs+1 x^0+1−αˉτs+1 ϵ^\mathbf{x}_{\tau_{s+1}} = \sqrt{\bar{\alpha}_{\tau_{s+1}}}\,\hat{\mathbf{x}}_0 + \sqrt{1 - \bar{\alpha}_{\tau_{s+1}}}\,\hat{\boldsymbol{\epsilon}}
  1. Return x^0\hat{\mathbf{x}}_0 at the final step.

Read step 3 as: guess the clean image, then re-noise that guess to exactly the next (lower) noise level — using the same predicted noise direction rather than a fresh random draw.

Step through it

The trace runs on a Gaussian mixture, whose optimal noise predictor is available in closed form, so there is no network to train, and no model error mixed into the measurement.

DDIM — the guess, the re-noise, and what step-skipping costs

1/6

The data is a three-mode Gaussian mixture, chosen because its **optimal noise predictor is available in closed form** — no network needs training, and the marginal at every noise level is itself a mixture. That matters for what follows: with the ideal ε in hand, everything measured below is the *sampler's* discretisation error, with no model error mixed in. Sampling starts from pure noise at t = 999, where ᾱ = 4.04e-5 — the signal is gone.

python
1x = randn_like(x0) # x_{tau_1}
2
3for s in range(S - 1):
4 t, t_next = tau[s], tau[s+1]
5
6 # the only network call
7 eps = model(x, t)
8
9 # 1. guess the clean image
10 x0 = (x - sqrt(1 - ab[t]) * eps)
11 x0 = x0 / sqrt(ab[t])
12
13 # 2. re-noise that guess to the
14 # next level, same direction
15 x = (sqrt(ab[t_next]) * x0
16 + sqrt(1 - ab[t_next]) * eps)
17
18return x0
state
the data distribution, and the starting noise
-3.23.23.2-3.2μ1μ2μ3x_Tdim 0dim 1
schedule
T → 1000ᾱ at t=0 → 0.9999ᾱ at t=999 → 4.04e-5steps this run → 10

Run on a Gaussian mixture, whose optimal noise predictor is available in closed form — so there is no network to train and no model error mixed into the measurement. Watch x̂₀, the model's running guess at the clean sample, start near the distribution mean and commit to one mode as the responsibilities collapse. Determinism is checked directly and the interpolation property measured as a path-length ratio. The payoff measures the page's ~20x speedup claim against a 1000-step reference from the same starting noise — error falls to 0.027 at 50 steps — and then turns η back up to 1, where two runs from the same start diverge instead of converging, because there is no trajectory to converge to.

What to notice:

  • Frame 2: the first guess is worthless, and that is fine. At t=999t=999 the clean-image estimate x^0\hat{x}_0 sits 1.35 from the nearest mode with responsibilities spread across all three. From pure noise the best possible estimate is near the distribution's mean. What makes the step work is not the guess's quality but that it is re-noised using the same ϵ^\hat\epsilon rather than a fresh draw.
  • Frame 3: follow x^0\hat{x}_0, not xx. The running guess migrates from the distribution mean onto one mode as responsibilities collapse (1.35 → 0.08, max responsibility 0.49 → 1.00). The commitment happens early: by the halfway point the outcome is settled.
  • Frame 5: the speedup claim, measured. Against a 1000-step reference from the same starting noise, endpoint error falls 1.651 at 2 steps to 0.027 at 50, monotonically. That is the ODE argument working: a deterministic trajectory integrated on a coarser grid.

The trace also measures what η\eta costs, and it is not sample quality. With a perfect ϵ\epsilon both samplers target the right distribution; what η>0\eta > 0 destroys is that the endpoint is a function of the start. Two η=1\eta = 1 runs from an identical starting noise land 2.14 apart at 50 steps. Semantic interpolation and reproducible seeds both depend on η=0\eta = 0.

Properties of DDIM

  • ~20× speedup: reducing T=1000T=1000 to S=50S=50 steps loses little quality.
  • Deterministic: the same starting noise xτ1\mathbf{x}_{\tau_1} always produces the same image. This enables semantic interpolation: interpolate between two starting noises and the outputs interpolate smoothly.
  • Same trained model: DDIM is a sampling procedure; it reuses a standard DDPM-trained noise predictor with no retraining.

The η knob

The full DDIM formulation has a parameter η∈[0,1]\eta \in [0, 1] interpolating between deterministic DDIM (η=0\eta = 0, shown above) and stochastic DDPM (η=1\eta = 1). Modern samplers (DPM-Solver, UniPC) push the same ODE view further with higher-order solvers, reaching good samples in 10–20 steps.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Referenced by
DDIM SamplingDiffusion ModelsVariational Autoencoders