ARIMA Order Selection: Box-Jenkins MethodologyOpen in Colab

ARIMA Order Selection: Box-Jenkins Methodology

The Box-Jenkins methodology is a four-phase iterative procedure for fitting an ARIMA model. Each phase produces evidence that feeds back into the next iteration if the model is misspecified.

The four phases of the procedure

  1. Identification: determine dd (differencing order) from the ADF test; read pp and qq from ACF/PACF plots.
  2. Estimation: fit the model via maximum likelihood to estimate ϕ1,…,ϕp\phi_1,\ldots,\phi_p and θ1,…,θq\theta_1,\ldots,\theta_q.
  3. Diagnostics: check that the residuals are white noise; if not, revise the order and repeat.
  4. Forecasting: use the fitted model to generate point forecasts and prediction intervals.

Determining the differencing order d with the ADF test

This is step 1a of Phase 1 (Identification).

The Augmented Dickey-Fuller (ADF) test tests the null hypothesis that the series has a unit root (is non-stationary). The test statistic for the regression Δyt=α+βt+γyt−1+∑jδjΔyt−j+εt\Delta y_t = \alpha + \beta t + \gamma y_{t-1} + \sum_{j}\delta_j \Delta y_{t-j} + \varepsilon_t is:

ADF=γ^/SE(γ^)\text{ADF} = \hat{\gamma} / \text{SE}(\hat{\gamma})

Decision rule: p-value < 0.05 → reject non-stationarity → proceed with current dd.

If p≥0.05p \ge 0.05, apply one more difference (d←d+1d \leftarrow d+1) and re-test. Typical decisions:

  • d=0d = 0: series is already stationary
  • d=1d = 1: first difference removes a linear trend
  • d=2d = 2: second difference removes a quadratic trend; rarely needed

Over-differencing

Differencing beyond what is needed (d=2 when d=1 suffices) inflates MA terms and makes estimation harder. Apply the minimum number of differences that passes the ADF test.

Reading p and q from the ACF/PACF

This is step 1b of Phase 1. After differencing dd times, inspect the correlogram of Δdyt\Delta^d y_t:

ACFPACFModel
Exponential/sinusoidal decayCuts off after lag ppAR(pp)
Cuts off after lag qqExponential/sinusoidal decayMA(qq)
Both decayBoth decayARMA(pp,qq): try small values
All within bandsAll within bandsWhite noise: dd was enough

For AR(pp): look at the lag where the PACF drops inside the ±1.96/T\pm 1.96/\sqrt{T} band. That lag is pp.

For MA(qq): look at the lag where the ACF drops inside the band. That lag is qq.

Parameter estimation and comparing candidates

This is Phase 2 (Estimation). Given tentative orders (p,d,q)(p, d, q), maximize the log-likelihood of the model:

yt=c+ϕ1yt−1+⋯+ϕpyt−p+εt+θ1εt−1+⋯+θqεt−qy_t = c + \phi_1 y_{t-1} + \cdots + \phi_p y_{t-p} + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \cdots + \theta_q \varepsilon_{t-q}

where εt∼N(0,σ2)\varepsilon_t \sim \mathcal{N}(0, \sigma^2). For the AR part, OLS is equivalent to MLE. For the MA part and mixed ARMA, numerical optimization (e.g., L-BFGS) is required because the likelihood involves the unobservable εt\varepsilon_t.

AIC and BIC for model comparison. Fit several candidate models and choose the one with the lowest information criterion:

AIC=−2ln⁡L^+2(p+q+1),BIC=−2ln⁡L^+(p+q+1)ln⁡T\text{AIC} = -2\ln\hat{\mathcal{L}} + 2(p+q+1), \quad \text{BIC} = -2\ln\hat{\mathcal{L}} + (p+q+1)\ln T

BIC penalizes extra parameters more strongly than AIC, so it prefers simpler models. Both should point to the same winner; if they disagree, prefer BIC for longer series.

Checking the residuals with diagnostics

This is Phase 3 (Diagnostics).

Ljung-Box test on residuals

The Ljung-Box test checks whether the first hh autocorrelations of the residuals are jointly zero:

Qh=T(T+2)∑k=1hρ^k2T−k  ∼  χ2(h−p−q)Q_h = T(T+2)\sum_{k=1}^{h}\frac{\hat{\rho}_k^2}{T-k} \;\sim\; \chi^2(h - p - q)

Want: p-value>0.05p\text{-value} > 0.05 (cannot reject white noise). A small pp-value means the residuals are still autocorrelated — the model order needs to increase.

Other checks:

  • ACF of residuals: all bars inside ±1.96/T\pm 1.96/\sqrt{T}
  • Q-Q plot: residuals close to the diagonal confirms Gaussian errors
  • Variance over time: no ARCH effects (conditional heteroscedasticity)

The whole procedure on a 36-observation series

Dataset (simulated monthly sales, Jan 2020 – Dec 2022). It is a draw from an actual ARIMA(0,1,1)-with-drift process (θ1=−0.6\theta_1 = -0.6, drift 2.0, σ=3\sigma = 3), rounded to integers, so the procedure below has something real to recover:

Code
Period:  1   2   3   4   5   6   7   8   9  10  11  12
Values: 50  51  54  55  60  56  64  63  65  64  63  67
Period: 13  14  15  16  17  18  19  20  21  22  23  24
Values: 72  73  77  77  80  81  80  84  85  90  90  92
Period: 25  26  27  28  29  30  31  32  33  34  35  36
Values: 96  99  99 101 104 109 106 106 119 115 118 114

ADF test on raw series (upward trend visible):

Code
ADF statistic: 2.55   p-value: 0.999   → non-stationary (fail to reject)

Apply d=1 differencing: Δyt\Delta y_t = [1, 3, 1, 5, −4, 8, −1, 2, −1, −1, 4, 5, 1, 4, 0, 3, 1, −1, 4, 1, 5, 0, 2, 4, 3, 0, 2, 3, 5, −3, 0, 13, −4, 3, −4]

ADF on Δyt\Delta y_t:

Code
ADF statistic: −3.86   p-value: 0.002   → stationary  ✓  →  d = 1

Mean of Δyt\Delta y_t = 1.83 (average monthly growth); its variance is 11.85, against 436.6 for the undifferenced series and 35.2 if you difference again — the variance is U-shaped in dd, which is a cheaper and more informative way to choose dd than a p-value alone.

ACF of Δyt\Delta y_t (band ±1.96/35=±0.331\pm 1.96/\sqrt{35} = \pm 0.331): ρ^1=−0.486\hat\rho_1 = −0.486, ρ^2=0.096\hat\rho_2 = 0.096, ρ^3=−0.137\hat\rho_3 = −0.137, ρ^4=0.041\hat\rho_4 = 0.041 — cuts off after lag 1.

PACF of Δyt\Delta y_t: ϕ^11=−0.500\hat\phi_{11} = −0.500, ϕ^22=−0.198\hat\phi_{22} = −0.198, ϕ^33=−0.260\hat\phi_{33} = −0.260, ϕ^44=−0.216\hat\phi_{44} = −0.216 — tails off.

→ ACF cuts at 1, PACF tails: suggests MA(1), i.e. ARIMA(0, 1, 1).

Candidate model comparison (statsmodels, trend='t' so the model carries the drift):

ModelAICBICLjung-Box p
ARIMA(0,1,1)173.3178.00.89 ✓
ARIMA(1,1,1)175.3181.50.89 ✓
ARIMA(0,1,2)175.3181.50.89 ✓
ARIMA(1,1,0)180.7185.30.83 ✓

Winner: ARIMA(0,1,1): lowest AIC and BIC, residuals are white noise.

Estimated parameters: μ^=1.96\hat\mu = 1.96 (drift), θ^1=−0.846\hat\theta_1 = −0.846, σ^=2.59\hat\sigma = 2.59.

Note how narrow the win is: ARIMA(1,1,1) and ARIMA(0,1,2) both nest ARIMA(0,1,1), and both come in exactly 2.0 AIC points behind. That is not a coincidence — see the trace below.

6-step forecast from t=36t = 36:

y^37=119.6,y^38=121.6,y^39=123.5,…,y^42=129.4\hat{y}_{37} = 119.6,\quad \hat{y}_{38} = 121.6,\quad \hat{y}_{39} = 123.5,\quad \ldots,\quad \hat{y}_{42} = 129.4

The forecast jumps 5.6 on the first step even though the drift is only 1.96, because θ^1=−0.846\hat\theta_1 = -0.846 makes the model treat the last observation's fall (118 → 114) as a shock to be partly reversed. After that it grows by the drift.

The 95% prediction intervals are ±5.1\pm 5.1 at h=1h=1 and only ±5.4\pm 5.4 at h=6h=6 — they barely widen. For ARIMA(0,1,1) the hh-step variance is σ2[1+(h−1)(1+θ1)2]\sigma^2[1 + (h-1)(1+\theta_1)^2], and with θ1≈−0.85\theta_1 \approx -0.85 the factor (1+θ1)2≈0.02(1+\theta_1)^2 \approx 0.02 is nearly zero. A random walk's intervals grow like h\sqrt{h}; an MA(1) with θ\theta near −1-1 has almost cancelled the differencing, so its uncertainty is nearly horizon-independent.

ARIMA with d ≥ 1 has no drift term by default

In statsmodels, ARIMA(y, order=(0,1,1)) fits no constant when d≥1d \ge 1. On this series that model forecasts a flat 114.3 forever (below the last observation), and scores AIC 197.4 instead of 173.3. If your series has a visible trend, you must pass trend='t', or you will silently fit a driftless model and wonder why the forecast is flat.

Running the whole workflow in Python

Python
import numpy as np
from statsmodels.tsa.stattools import adfuller
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.stats.diagnostic import acorr_ljungbox
import warnings
warnings.filterwarnings("ignore")

# Example series
y = np.array([50,51,54,55,60,56,64,63,65,64,63,67,
              72,73,77,77,80,81,80,84,85,90,90,92,
              96,99,99,101,104,109,106,106,119,115,118,114], dtype=float)

# Phase 1: determine d
print("ADF raw:", adfuller(y)[1].round(4))            # 0.9991 → non-stationary
print("ADF diff:", adfuller(np.diff(y))[1].round(4))  # 0.0023 → d=1

# Phase 2 + 3: fit candidates, compare AIC/BIC, check residuals
# trend='t' keeps the drift; without it, d>=1 fits no constant at all
for order in [(0,1,1), (1,1,1), (0,1,2), (1,1,0)]:
    res = ARIMA(y, order=order, trend='t').fit()
    lb = acorr_ljungbox(res.resid, lags=10, return_df=True)["lb_pvalue"].min()
    print(f"ARIMA{order}: AIC={res.aic:.1f}  BIC={res.bic:.1f}  LjungBox_min_p={lb:.3f}")

# Phase 4: forecast
best = ARIMA(y, order=(0,1,1), trend='t').fit()
forecast = best.get_forecast(steps=6)
print("\n6-step forecast:")
print(forecast.summary_frame(alpha=0.05)[["mean","mean_ci_lower","mean_ci_upper"]])

Step through the selection

ARIMA order selection — the AIC grid, and how little it wins by

1/8

36 monthly observations, rising from 50 to 114. Before fitting anything, the identification phase has to answer one question: is this stationary? The autocorrelations of the raw series answer it — 0.92, 0.83, 0.75, 0.65 — decaying slowly and staying positive for many lags rather than dying out. That slow decay is the signature of a **unit root**: the series has no level it returns to, so its mean depends on where you start looking.

python
1# 1. identification: choose d
2d = 0
3while not stationary(diff(y, d)):
4 d += 1
5z = diff(y, d)
6
7# 2. estimation: grid search by AIC
8best = None
9for p, q in product(range(3), range(3)):
10 fit = css_arma(z, p, q)
11 k = p + q + 2
12 aic = -2 * fit.loglike + 2 * k
13 if best is None or aic < best.aic:
14 best = fit
15
16# 3. diagnostics: white-noise residuals
17ljung_box(best.resid, lags=10)
state
the series (T = 36)
13612540monthsales
ACF of the raw series — persistent, not decaying to zero
ρ1
0.92
ρ2
0.83
ρ3
0.75
ρ4
0.65
ρ5
0.59
ρ6
0.51

Box-Jenkins end to end on 36 observations: the raw ACF's slow decay identifies a unit root, the variance of the differenced series picks d by a visible U-shape, the correlogram of Δy proposes MA(1), and a 9-model AIC grid confirms it. Then two measured payoffs. Over-differencing raises the variance 3x and pushes θ̂ toward the −1 boundary — the unit MA root the extra difference itself created. And the winner's AIC margin over its nesting rival is 1.96, which is not a near-miss but a ceiling: when the extra parameter is useless the entire difference is the penalty constant, so AIC can never prefer the simpler nested model by more than 2. BIC's ln T = 3.56 makes the same call twice as decisively.

What to notice:

  • Frame 2: choosing dd without a hypothesis test. The variance of Δdy\Delta^d y is 436.6, 11.9, 35.2, 120.4 for d=0,1,2,3d = 0,1,2,3. The minimum is the answer, and the U-shape shows why over-differencing is a real cost rather than a stylistic preference.
  • Frame 7: the over-differencing payoff. Differencing twice triples the variance and pushes θ^1\hat\theta_1 from −0.84-0.84 to −0.99-0.99. That is not coincidence: differencing an already-stationary series applies an MA filter with a unit root θ=−1\theta = -1, so the estimator must spend a parameter cancelling a root the analyst introduced, right on the boundary of the parameter space where estimation behaves worst.
  • Frame 8: why the margin is 2.00. Every model that nests ARIMA(0,1,1) comes in at +1.96+1.96, +3.92+3.92, +5.61+5.61 AIC — very close to 2k2k for kk redundant parameters. When the extra coefficient is useless the log-likelihood does not move, so the entire gap is the penalty term. AIC can never prefer the simpler nested model by more than 2 per parameter, which is why a 2-point AIC win is weak evidence and BIC's ln⁡T=3.56\ln T = 3.56 penalty is the sharper instrument.

The trace estimates by conditional sum of squares rather than exact MLE, so its absolute AIC values sit about 0.9 below statsmodels'. The ranking is identical across all nine candidates.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
ARIMA Order Selection: Box-Jenkin…ACF & PACF: Reading the C…Walk-Forward ValidationHierarchical Forecasting …The Augmented Dickey-Full…ARIMA Models