Walk-Forward ValidationOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Walk-Forward Validation

Standard k-fold cross-validation randomly shuffles data across folds. For time series this is invalid: a training point at t=80t = 80 could end up alongside a validation point at t=60t = 60, so the model is trained on data it would never have seen at prediction time. Walk-forward validation preserves temporal order by always training on the past and validating on strict future windows.

Why k-fold cross-validation leaks the future

Consider a 10-observation series with k-fold k=5 (2 observations per fold):

Code
Fold assignment (random):  4 1 3 2 5 1 4 2 5 3
Observation:               1 2 3 4 5 6 7 8 9 10

When fold 1 is the validation set, the training set includes fold 3 (observations 3 and 10), fold 4 (1 and 7), fold 5 (5 and 9). The model trained on observations [3, 5, 7, 9, 10] is asked to predict observation [2]. It has seen the future of observation 2. Any temporal signal (trends, autocorrelation) gets leaked from future to past.

The two walk-forward schemes

Expanding window

Each fold adds one more observation to the training set:

Code
Round 1:  train [1..n₀]       → validate [n₀+1 .. n₀+h]
Round 2:  train [1..n₀+1]     → validate [n₀+2 .. n₀+h+1]
Round 3:  train [1..n₀+2]     → validate [n₀+3 .. n₀+h+2]
…

Best for: when more data is always better (most forecasting scenarios). Mimics real deployment: the model sees all available history.

Sliding window

The training window has fixed size WW; it slides forward each round:

Code
Round 1:  train [1..W]         → validate [W+1 .. W+h]
Round 2:  train [2..W+1]       → validate [W+2 .. W+h+1]
Round 3:  train [3..W+2]       → validate [W+3 .. W+h+2]
…

Best for: when the process is non-stationary and very old data is irrelevant (e.g., financial series where market regimes change). Avoids the training set growing without bound.

A one-step expanding-window evaluation by hand

Worked trace: h=1h=1, expanding window.

Series (12 observations): [10,12,14,13,15,17,16,18,20,19,21,23][10, 12, 14, 13, 15, 17, 16, 18, 20, 19, 21, 23]

Minimum training size n0=6n_0 = 6. Predictor: mean of the 3 most recent observations (simple but concrete).

| Round | Train | Predict tt | y^t=mean(yt−1,yt−2,yt−3)\hat{y}_t = \text{mean}(y_{t-1}, y_{t-2}, y_{t-3}) | yty_t (actual) | ∣et∣|e_t| | |-------|-------|-------------|------------------------------------------------------|----------------|---------| | 1 | 1–6 | 7 | mean(17, 15, 13) = 15.00 | 16 | 1.00 | | 2 | 1–7 | 8 | mean(16, 17, 15) = 16.00 | 18 | 2.00 | | 3 | 1–8 | 9 | mean(18, 16, 17) = 17.00 | 20 | 3.00 | | 4 | 1–9 | 10 | mean(20, 18, 16) = 18.00 | 19 | 1.00 | | 5 | 1–10 | 11 | mean(19, 20, 18) = 19.00 | 21 | 2.00 | | 6 | 1–11 | 12 | mean(21, 19, 20) = 20.00 | 23 | 3.00 |

MAECV=1+2+3+1+2+36=126=2.00\text{MAE}_\text{CV} = \frac{1+2+3+1+2+3}{6} = \frac{12}{6} = 2.00

This is the walk-forward estimate of the model's one-step-ahead MAE. Compare to, say, a linear trend predictor to decide which model to deploy.

Step through it

The trace runs exactly the table above, fold by fold, then shows what random k-fold does to the model choice.

Walk-forward validation — and what random k-fold selects instead

1/10

The page's series: 12 observations, minimum training size n₀ = 6, horizon h = 1, and a predictor that averages the 3 most recent training points. One rule governs everything below — **the training window may only contain observations strictly before the point being predicted.** Colour code for the rest of the trace: grey is in the training window, bright is what the predictor actually reads, amber is the point being predicted, and red is the future the fold is forbidden to look at.

python
1def walk_forward(y, fit, n0, h, window):
2 errors = []
3 for t in range(n0, len(y) - h + 1):
4
5 # train strictly on the past
6 s = t - window if window else 0
7 model = fit(y[s:t])
8
9 # predict a strictly future point
10 yhat = model.predict(h)
11 e = abs(y[t : t + h] - yhat)
12 errors.append(e)
13
14 return mean(errors)
state
series (n₀ = 6, so round 1 predicts t7)
10t1
12t2
14t3
13t4
15t5
17t6
16t7
18t8
20t9
19t10
21t11
23t12
configuration
T → 12n₀ → 6h → 1rounds → 6window → expanding

The page's own 12-point example, run fold by fold to its MAE of 2.00, with the per-fold errors showing a 1-2-3-1-2-3 pattern that the average hides — a backward-looking predictor lagging a trend. Then the part the page asserts but cannot show: score eight polynomial trends with random k-fold and with walk-forward and the two error curves run in opposite directions, because one asks the model to interpolate and the other to extrapolate. k-fold picks degree 6-8 in all 20 seeds tried; walk-forward picks 1-2; on a genuinely held-out future, k-fold's choice loses every time.

What to notice:

  • Frames 2–7: the error pattern the average destroys. The per-fold errors are 1, 2, 3, 1, 2, 3. That is not noise: a backward-looking average lags a rising series and falls further behind each step the trend continues, then recovers at the dip on t10t_{10}. The single number 2.00 hides a systematic bias that a residual plot would have caught immediately.
  • Frame 9: expanding vs sliding needs a predictor that can tell the difference. The mean-of-3 predictor above reads three points regardless of window size, so both schemes give the same answer. Swap in a predictor that uses its whole window and sliding (W=6W=6, MAE 3.50) beats expanding (4.83) — on a trending series old observations do not just fail to help, they drag the average down.
  • Frame 10: the payoff. Score eight polynomial trends with random 5-fold and with walk-forward: k-fold's error falls with degree (1.97 → 1.55) while walk-forward's rises (2.15 → 3.53). The curves run in opposite directions because random folds ask the model to interpolate between neighbouring points, which rewards wiggle, and walk-forward asks it to extrapolate, which punishes it. Across 20 seeds k-fold picks degree 6–8 every time, walk-forward picks 1–2 every time, and on a genuinely held-out future k-fold's choice loses 20/20 — by 33× to 849×.

The last point is worth stating precisely, because "k-fold is wrong here" is not quite it: k-fold's numbers are correctly computed and are a valid estimate of interpolation error. That is simply not the quantity a deployed forecaster is judged on.

Overlap and independence

When the forecast horizon h=1h = 1, each validation window touches a different time point — the folds are non-overlapping and the errors are approximately independent. When h>1h > 1, adjacent windows share observations (window 1 predicts t=7t=7; window 2 predicts t=7t=7 and t=8t=8), introducing correlation between fold errors. Report the average anyway, but interpret the standard error of CV error cautiously.

Choosing the initial training size n0n_0

  • At minimum, n0n_0 must be large enough to fit the model (e.g., at least p+q+d+1p + q + d + 1 observations for ARIMA, or several windows for LSTM).
  • Larger n0n_0 gives more stable first estimates but fewer validation rounds.
  • Common choice: n0=0.5Tn_0 = 0.5T to 0.7T0.7T, leaving 30–50% of the series for evaluation.

Implementing walk-forward validation in Python

Python
import numpy as np
from typing import Callable

def walk_forward_cv(
    y: np.ndarray,
    model_fn: Callable,       # takes y_train, returns a predictor function
    n0: int = None,           # initial training size
    h: int = 1,               # forecast horizon
    window: int = None,       # if set, use sliding window of this size
) -> dict:
    """Walk-forward cross-validation for time series."""
    T = len(y)
    if n0 is None:
        n0 = max(T // 2, 10)

    errors = []
    for t in range(n0, T - h + 1):
        train_start = max(0, t - window) if window else 0
        y_train = y[train_start:t]
        y_val   = y[t : t + h]

        predictor = model_fn(y_train)
        y_hat = predictor(h)
        errors.append(np.abs(y_val - y_hat))

    errors = np.array(errors)
    return {
        "mae":  errors.mean(),
        "rmse": np.sqrt((errors**2).mean()),
        "n_folds": len(errors),
    }

# Example: naive mean-of-3 predictor
def naive_mean3(y_train):
    last3 = y_train[-3:].mean()
    def predict(h):
        return np.full(h, last3)
    return predict

y = np.array([10,12,14,13,15,17,16,18,20,19,21,23], dtype=float)
result = walk_forward_cv(y, naive_mean3, n0=6, h=1)
print(f"Walk-forward MAE: {result['mae']:.2f}")   # → 2.00
print(f"Folds evaluated: {result['n_folds']}")    # → 6

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
Walk-Forward ValidationARIMA Order Selection: Bo…Hierarchical Forecasting …The ML Project LoopBias-Variance Decompositi…Deep Learning for Time Se…