DDIM Sampling
Standard DDPM sampling requires walking back through all noise levels — one network call per step. DDIM (Denoising Diffusion Implicit Models, Song et al., 2021) reformulates the reverse process as a deterministic ODE, which makes step-skipping valid: you can visit 50 of the 1000 timesteps and still land on a high-quality sample.
Why step-skipping is valid
DDPM's reverse process is stochastic — each step adds fresh noise, so the trajectory is a random walk that must be taken one step at a time. DDIM removes the stochasticity: given the model's noise prediction, both the "clean image estimate" and the next point on the trajectory are deterministic functions of the current point. A deterministic trajectory can be integrated with large steps, exactly like solving an ODE with a coarser grid.
The DDIM sampling procedure
Recall the cumulative noise coefficient , where is the noise variance added at step . Because is small, decreases from (clean) to (pure noise) as increases from to .
- Choose a subsequence of timesteps to visit (e.g. every 20th step for ).
- Start from pure noise .
- At each step (going from down to ):
- Run the network to predict the noise: .
- Estimate the clean image:
- Step to the next timestep by re-noising the estimate to the target noise level:
- Return at the final step.
Read step 3 as: guess the clean image, then re-noise that guess to exactly the next (lower) noise level — using the same predicted noise direction rather than a fresh random draw.
Step through it
The trace runs on a Gaussian mixture, whose optimal noise predictor is available in closed form, so there is no network to train, and no model error mixed into the measurement.
DDIM — the guess, the re-noise, and what step-skipping costs
The data is a three-mode Gaussian mixture, chosen because its **optimal noise predictor is available in closed form** — no network needs training, and the marginal at every noise level is itself a mixture. That matters for what follows: with the ideal ε in hand, everything measured below is the *sampler's* discretisation error, with no model error mixed in. Sampling starts from pure noise at t = 999, where ᾱ = 4.04e-5 — the signal is gone.
Run on a Gaussian mixture, whose optimal noise predictor is available in closed form — so there is no network to train and no model error mixed into the measurement. Watch x̂₀, the model's running guess at the clean sample, start near the distribution mean and commit to one mode as the responsibilities collapse. Determinism is checked directly and the interpolation property measured as a path-length ratio. The payoff measures the page's ~20x speedup claim against a 1000-step reference from the same starting noise — error falls to 0.027 at 50 steps — and then turns η back up to 1, where two runs from the same start diverge instead of converging, because there is no trajectory to converge to.
What to notice:
- Frame 2: the first guess is worthless, and that is fine. At the clean-image estimate sits 1.35 from the nearest mode with responsibilities spread across all three. From pure noise the best possible estimate is near the distribution's mean. What makes the step work is not the guess's quality but that it is re-noised using the same rather than a fresh draw.
- Frame 3: follow , not . The running guess migrates from the distribution mean onto one mode as responsibilities collapse (1.35 → 0.08, max responsibility 0.49 → 1.00). The commitment happens early: by the halfway point the outcome is settled.
- Frame 5: the speedup claim, measured. Against a 1000-step reference from the same starting noise, endpoint error falls 1.651 at 2 steps to 0.027 at 50, monotonically. That is the ODE argument working: a deterministic trajectory integrated on a coarser grid.
The trace also measures what costs, and it is not sample quality. With a perfect both samplers target the right distribution; what destroys is that the endpoint is a function of the start. Two runs from an identical starting noise land 2.14 apart at 50 steps. Semantic interpolation and reproducible seeds both depend on .
Properties of DDIM
- ~20× speedup: reducing to steps loses little quality.
- Deterministic: the same starting noise always produces the same image. This enables semantic interpolation: interpolate between two starting noises and the outputs interpolate smoothly.
- Same trained model: DDIM is a sampling procedure; it reuses a standard DDPM-trained noise predictor with no retraining.
The η knob
The full DDIM formulation has a parameter interpolating between deterministic DDIM (, shown above) and stochastic DDPM (). Modern samplers (DPM-Solver, UniPC) push the same ODE view further with higher-order solvers, reaching good samples in 10–20 steps.
Related concepts
- Diffusion Models: the forward/reverse processes and training objective
- Variational Autoencoders: the ELBO machinery diffusion training inherits