Walk-Forward Validation
Standard k-fold cross-validation randomly shuffles data across folds. For time series this is invalid: a training point at could end up alongside a validation point at , so the model is trained on data it would never have seen at prediction time. Walk-forward validation preserves temporal order by always training on the past and validating on strict future windows.
Why k-fold cross-validation leaks the future
Consider a 10-observation series with k-fold k=5 (2 observations per fold):
Fold assignment (random): 4 1 3 2 5 1 4 2 5 3
Observation: 1 2 3 4 5 6 7 8 9 10
When fold 1 is the validation set, the training set includes fold 3 (observations 3 and 10), fold 4 (1 and 7), fold 5 (5 and 9). The model trained on observations [3, 5, 7, 9, 10] is asked to predict observation [2]. It has seen the future of observation 2. Any temporal signal (trends, autocorrelation) gets leaked from future to past.
The two walk-forward schemes
Expanding window
Each fold adds one more observation to the training set:
Round 1: train [1..n₀] → validate [n₀+1 .. n₀+h]
Round 2: train [1..n₀+1] → validate [n₀+2 .. n₀+h+1]
Round 3: train [1..n₀+2] → validate [n₀+3 .. n₀+h+2]
…
Best for: when more data is always better (most forecasting scenarios). Mimics real deployment: the model sees all available history.
Sliding window
The training window has fixed size ; it slides forward each round:
Round 1: train [1..W] → validate [W+1 .. W+h]
Round 2: train [2..W+1] → validate [W+2 .. W+h+1]
Round 3: train [3..W+2] → validate [W+3 .. W+h+2]
…
Best for: when the process is non-stationary and very old data is irrelevant (e.g., financial series where market regimes change). Avoids the training set growing without bound.
A one-step expanding-window evaluation by hand
Worked trace: , expanding window.
Series (12 observations):
Minimum training size . Predictor: mean of the 3 most recent observations (simple but concrete).
| Round | Train | Predict | | (actual) | | |-------|-------|-------------|------------------------------------------------------|----------------|---------| | 1 | 1–6 | 7 | mean(17, 15, 13) = 15.00 | 16 | 1.00 | | 2 | 1–7 | 8 | mean(16, 17, 15) = 16.00 | 18 | 2.00 | | 3 | 1–8 | 9 | mean(18, 16, 17) = 17.00 | 20 | 3.00 | | 4 | 1–9 | 10 | mean(20, 18, 16) = 18.00 | 19 | 1.00 | | 5 | 1–10 | 11 | mean(19, 20, 18) = 19.00 | 21 | 2.00 | | 6 | 1–11 | 12 | mean(21, 19, 20) = 20.00 | 23 | 3.00 |
This is the walk-forward estimate of the model's one-step-ahead MAE. Compare to, say, a linear trend predictor to decide which model to deploy.
Step through it
The trace runs exactly the table above, fold by fold, then shows what random k-fold does to the model choice.
Walk-forward validation — and what random k-fold selects instead
The page's series: 12 observations, minimum training size n₀ = 6, horizon h = 1, and a predictor that averages the 3 most recent training points. One rule governs everything below — **the training window may only contain observations strictly before the point being predicted.** Colour code for the rest of the trace: grey is in the training window, bright is what the predictor actually reads, amber is the point being predicted, and red is the future the fold is forbidden to look at.
The page's own 12-point example, run fold by fold to its MAE of 2.00, with the per-fold errors showing a 1-2-3-1-2-3 pattern that the average hides — a backward-looking predictor lagging a trend. Then the part the page asserts but cannot show: score eight polynomial trends with random k-fold and with walk-forward and the two error curves run in opposite directions, because one asks the model to interpolate and the other to extrapolate. k-fold picks degree 6-8 in all 20 seeds tried; walk-forward picks 1-2; on a genuinely held-out future, k-fold's choice loses every time.
What to notice:
- Frames 2–7: the error pattern the average destroys. The per-fold errors are 1, 2, 3, 1, 2, 3. That is not noise: a backward-looking average lags a rising series and falls further behind each step the trend continues, then recovers at the dip on . The single number 2.00 hides a systematic bias that a residual plot would have caught immediately.
- Frame 9: expanding vs sliding needs a predictor that can tell the difference. The mean-of-3 predictor above reads three points regardless of window size, so both schemes give the same answer. Swap in a predictor that uses its whole window and sliding (, MAE 3.50) beats expanding (4.83) — on a trending series old observations do not just fail to help, they drag the average down.
- Frame 10: the payoff. Score eight polynomial trends with random 5-fold and with walk-forward: k-fold's error falls with degree (1.97 → 1.55) while walk-forward's rises (2.15 → 3.53). The curves run in opposite directions because random folds ask the model to interpolate between neighbouring points, which rewards wiggle, and walk-forward asks it to extrapolate, which punishes it. Across 20 seeds k-fold picks degree 6–8 every time, walk-forward picks 1–2 every time, and on a genuinely held-out future k-fold's choice loses 20/20 — by 33× to 849×.
The last point is worth stating precisely, because "k-fold is wrong here" is not quite it: k-fold's numbers are correctly computed and are a valid estimate of interpolation error. That is simply not the quantity a deployed forecaster is judged on.
Overlap and independence
When the forecast horizon , each validation window touches a different time point — the folds are non-overlapping and the errors are approximately independent. When , adjacent windows share observations (window 1 predicts ; window 2 predicts and ), introducing correlation between fold errors. Report the average anyway, but interpret the standard error of CV error cautiously.
Choosing the initial training size
- At minimum, must be large enough to fit the model (e.g., at least observations for ARIMA, or several windows for LSTM).
- Larger gives more stable first estimates but fewer validation rounds.
- Common choice: to , leaving 30–50% of the series for evaluation.
Implementing walk-forward validation in Python
import numpy as np
from typing import Callable
def walk_forward_cv(
y: np.ndarray,
model_fn: Callable, # takes y_train, returns a predictor function
n0: int = None, # initial training size
h: int = 1, # forecast horizon
window: int = None, # if set, use sliding window of this size
) -> dict:
"""Walk-forward cross-validation for time series."""
T = len(y)
if n0 is None:
n0 = max(T // 2, 10)
errors = []
for t in range(n0, T - h + 1):
train_start = max(0, t - window) if window else 0
y_train = y[train_start:t]
y_val = y[t : t + h]
predictor = model_fn(y_train)
y_hat = predictor(h)
errors.append(np.abs(y_val - y_hat))
errors = np.array(errors)
return {
"mae": errors.mean(),
"rmse": np.sqrt((errors**2).mean()),
"n_folds": len(errors),
}
# Example: naive mean-of-3 predictor
def naive_mean3(y_train):
last3 = y_train[-3:].mean()
def predict(h):
return np.full(h, last3)
return predict
y = np.array([10,12,14,13,15,17,16,18,20,19,21,23], dtype=float)
result = walk_forward_cv(y, naive_mean3, n0=6, h=1)
print(f"Walk-forward MAE: {result['mae']:.2f}") # → 2.00
print(f"Folds evaluated: {result['n_folds']}") # → 6
Related concepts
- Deep Learning for Time Series: the full lesson: LSTM, TCN, evaluation metrics
- ARIMA Order Selection: the classical modelling workflow this evaluation supports
- Bias-Variance Decomposition: cross-validation in the context of bias-variance tradeoff