Hierarchical Forecasting & ReconciliationOpen in Colab

Hierarchical Forecasting & Reconciliation

A hierarchical time series is a collection of series organized in an aggregation structure: individual SKU sales sum to product-category sales, which sum to regional sales, which sum to national totals. Forecasting each level independently produces incoherent forecasts — the summed level-2 forecasts don't equal the level-1 forecast. Reconciliation methods project the base forecasts onto the coherent subspace where all aggregation constraints hold.

The summing matrix

Let nn = total number of series across all levels, mm = number of bottom-level series. The summing matrix SS (n×mn \times m) encodes the aggregation structure:

yt=S bt\mathbf{y}_t = S\,\mathbf{b}_t

where bt∈Rm\mathbf{b}_t \in \mathbb{R}^m are the bottom-level series values and yt∈Rn\mathbf{y}_t \in \mathbb{R}^n are all series (top + intermediate + bottom).

Example: 1 total, 2 regions, 4 bottom-level stores (n=7, m=4):

S=(1111110000111000010000100001)S = \begin{pmatrix} 1 & 1 & 1 & 1 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{pmatrix}

A set of forecasts y^\hat{\mathbf{y}} is coherent if and only if y^=S b^\hat{\mathbf{y}} = S\,\hat{\mathbf{b}} for some b^\hat{\mathbf{b}}.

Approach 1: Top-down

Produce a forecast only at the top level y^(top)\hat{y}^{(top)}, then disaggregate using historical proportions pip_i (fraction of total attributable to series ii):

y^i=pi⋅y^(top)\hat{y}_i = p_i \cdot \hat{y}^{(top)}

Proportions options:

  • Average historical proportions: pi=1T∑tyi,tyt(top)p_i = \frac{1}{T}\sum_t \frac{y_{i,t}}{y^{(top)}_t}
  • Proportions from the last period
  • Forecasted proportions (using a separate model)

When to use: When bottom-level series are sparse or noisy and the aggregate is well-behaved.
Limitation: Forces bottom-level forecasts to inherit the top-level model's biases.

Approach 2: Bottom-up

Forecast each bottom-level series independently, then aggregate:

y^(top)=∑iy^i,y^(region)=∑i∈regiony^i\hat{y}^{(top)} = \sum_i \hat{y}_i, \qquad \hat{y}^{(region)} = \sum_{i \in \text{region}} \hat{y}_i

When to use: When bottom-level series have sufficient history and you trust the granular models.
Limitation: Aggregation propagates and amplifies bottom-level errors.

Approach 3: MinT (Minimum Trace) reconciliation

Optimal reconciliation finds the coherent forecast y~\tilde{\mathbf{y}} closest (in MSE) to the base forecasts y^\hat{\mathbf{y}}:

y~=S(S⊤W−1S)−1S⊤W−1y^\tilde{\mathbf{y}} = S(S^\top W^{-1} S)^{-1} S^\top W^{-1} \hat{\mathbf{y}}

where WW is the n×nn \times n covariance matrix of base forecast errors y^−y\hat{\mathbf{y}} - \mathbf{y}.

Derivation intuition: This is generalized least squares (GLS) projecting y^\hat{\mathbf{y}} onto the coherent subspace {y:y=Sb}\{\mathbf{y} : \mathbf{y} = S\mathbf{b}\}. The weighting by W−1W^{-1} ensures that high-variance forecasts are trusted less in the reconciliation.

Estimating W

Estimating the full n×nn \times n covariance from historical forecast errors is expensive for large hierarchies. Common approximations:

MethodWWComplexity
OLSII (identity)O(n)O(n): simple average
WLS (variance scaling)diag(σ^12,…,σ^n2)\text{diag}(\hat{\sigma}_1^2, \ldots, \hat{\sigma}_n^2)O(n)O(n): series-wise variance
MinT (shrinkage)Shrinkage estimator of full covarianceO(n2)O(n^2)
MinT (sample)Sample covariance from rolling residualsO(Tn2)O(T n^2)

For large hierarchies (n>1000n > 1000), WLS with variance scaling is the practical choice; it captures most of the gain from MinT at a fraction of the cost.

Worked example

Setup: a minimal hierarchy — 1 national series that must equal the sum of 2 regional series. Each series is forecast independently, giving base predictions:

SeriesBase forecast (y^\hat{y})
National100
Region A58
Region B47

These are incoherent: the regions sum to 58+47=10558 + 47 = 105, but the independently forecast national total is 100100. They disagree by 5, and no single set of numbers is actionable until they are reconciled.

For this reduced hierarchy the summing matrix and base vector are

S=(111001),y^=(1005847).S = \begin{pmatrix} 1 & 1 \\ 1 & 0 \\ 0 & 1 \end{pmatrix}, \qquad \hat{\mathbf{y}} = \begin{pmatrix} 100 \\ 58 \\ 47 \end{pmatrix}.

MinT reconciliation with OLS (W=IW = I) projects y^\hat{\mathbf{y}} onto the coherent subspace via y~=Py^\tilde{\mathbf{y}} = P\hat{\mathbf{y}} with P=S(S⊤S)−1S⊤P = S(S^\top S)^{-1} S^\top. The pieces:

S⊤S=(2112),(S⊤S)−1=13(2−1−12),P=13(21112−11−12).S^\top S = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}, \quad (S^\top S)^{-1} = \frac{1}{3}\begin{pmatrix} 2 & -1 \\ -1 & 2 \end{pmatrix}, \quad P = \frac{1}{3}\begin{pmatrix} 2 & 1 & 1 \\ 1 & 2 & -1 \\ 1 & -1 & 2 \end{pmatrix}.

Applying the projection:

y~=P y^=13(21112−11−12)(1005847)=13(305169136)≈(101.756.345.3).\tilde{\mathbf{y}} = P\,\hat{\mathbf{y}} = \frac{1}{3}\begin{pmatrix} 2 & 1 & 1 \\ 1 & 2 & -1 \\ 1 & -1 & 2 \end{pmatrix} \begin{pmatrix} 100 \\ 58 \\ 47 \end{pmatrix} = \frac{1}{3}\begin{pmatrix} 305 \\ 169 \\ 136 \end{pmatrix} \approx \begin{pmatrix} 101.7 \\ 56.3 \\ 45.3 \end{pmatrix}.

The reconciled forecasts are now coherent (56.3+45.3=101.756.3 + 45.3 = 101.7) and each is the L2-closest coherent value to the base forecast. The reconciliation spread the 5-unit gap across all three series: the national estimate rose from 100 to 101.7 while both regions were pulled down slightly. Under OLS every series is trusted equally; WLS or full MinT would instead pull hardest on the highest-variance (least-trusted) forecasts.

Software

  • Python: statsforecast package provides MinT, TopDown, BottomUp via the HierarchicalReconciliation class.
  • R: fable package, reconcile() function with MinT() / ETS() methods.
  • Reference: Wickramasuriya et al. (2019) "Optimal Forecast Reconciliation Using Unbiased Mapping."

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
Hierarchical Forecasting & Reconc…ARIMA Order Selection: Bo…Walk-Forward ValidationDemand Forecasting in Pro…