ARIMA Order Selection: Box-Jenkins Methodology
The Box-Jenkins methodology is a four-phase iterative procedure for fitting an ARIMA model. Each phase produces evidence that feeds back into the next iteration if the model is misspecified.
The four phases of the procedure
- Identification: determine (differencing order) from the ADF test; read and from ACF/PACF plots.
- Estimation: fit the model via maximum likelihood to estimate and .
- Diagnostics: check that the residuals are white noise; if not, revise the order and repeat.
- Forecasting: use the fitted model to generate point forecasts and prediction intervals.
Determining the differencing order d with the ADF test
This is step 1a of Phase 1 (Identification).
The Augmented Dickey-Fuller (ADF) test tests the null hypothesis that the series has a unit root (is non-stationary). The test statistic for the regression is:
Decision rule: p-value < 0.05 → reject non-stationarity → proceed with current .
If , apply one more difference () and re-test. Typical decisions:
- : series is already stationary
- : first difference removes a linear trend
- : second difference removes a quadratic trend; rarely needed
Over-differencing
Differencing beyond what is needed (d=2 when d=1 suffices) inflates MA terms and makes estimation harder. Apply the minimum number of differences that passes the ADF test.
Reading p and q from the ACF/PACF
This is step 1b of Phase 1. After differencing times, inspect the correlogram of :
| ACF | PACF | Model |
|---|---|---|
| Exponential/sinusoidal decay | Cuts off after lag | AR() |
| Cuts off after lag | Exponential/sinusoidal decay | MA() |
| Both decay | Both decay | ARMA(,): try small values |
| All within bands | All within bands | White noise: was enough |
For AR(): look at the lag where the PACF drops inside the band. That lag is .
For MA(): look at the lag where the ACF drops inside the band. That lag is .
Parameter estimation and comparing candidates
This is Phase 2 (Estimation). Given tentative orders , maximize the log-likelihood of the model:
where . For the AR part, OLS is equivalent to MLE. For the MA part and mixed ARMA, numerical optimization (e.g., L-BFGS) is required because the likelihood involves the unobservable .
AIC and BIC for model comparison. Fit several candidate models and choose the one with the lowest information criterion:
BIC penalizes extra parameters more strongly than AIC, so it prefers simpler models. Both should point to the same winner; if they disagree, prefer BIC for longer series.
Checking the residuals with diagnostics
This is Phase 3 (Diagnostics).
Ljung-Box test on residuals
The Ljung-Box test checks whether the first autocorrelations of the residuals are jointly zero:
Want: (cannot reject white noise). A small -value means the residuals are still autocorrelated — the model order needs to increase.
Other checks:
- ACF of residuals: all bars inside
- Q-Q plot: residuals close to the diagonal confirms Gaussian errors
- Variance over time: no ARCH effects (conditional heteroscedasticity)
The whole procedure on a 36-observation series
Dataset (simulated monthly sales, Jan 2020 – Dec 2022). It is a draw from an actual ARIMA(0,1,1)-with-drift process (, drift 2.0, ), rounded to integers, so the procedure below has something real to recover:
Period: 1 2 3 4 5 6 7 8 9 10 11 12
Values: 50 51 54 55 60 56 64 63 65 64 63 67
Period: 13 14 15 16 17 18 19 20 21 22 23 24
Values: 72 73 77 77 80 81 80 84 85 90 90 92
Period: 25 26 27 28 29 30 31 32 33 34 35 36
Values: 96 99 99 101 104 109 106 106 119 115 118 114
ADF test on raw series (upward trend visible):
ADF statistic: 2.55 p-value: 0.999 → non-stationary (fail to reject)
Apply d=1 differencing: = [1, 3, 1, 5, −4, 8, −1, 2, −1, −1, 4, 5, 1, 4, 0, 3, 1, −1, 4, 1, 5, 0, 2, 4, 3, 0, 2, 3, 5, −3, 0, 13, −4, 3, −4]
ADF on :
ADF statistic: −3.86 p-value: 0.002 → stationary ✓ → d = 1
Mean of = 1.83 (average monthly growth); its variance is 11.85, against 436.6 for the undifferenced series and 35.2 if you difference again — the variance is U-shaped in , which is a cheaper and more informative way to choose than a p-value alone.
ACF of (band ): , , , — cuts off after lag 1.
PACF of : , , , — tails off.
→ ACF cuts at 1, PACF tails: suggests MA(1), i.e. ARIMA(0, 1, 1).
Candidate model comparison (statsmodels, trend='t' so the model carries the drift):
| Model | AIC | BIC | Ljung-Box p |
|---|---|---|---|
| ARIMA(0,1,1) | 173.3 | 178.0 | 0.89 ✓ |
| ARIMA(1,1,1) | 175.3 | 181.5 | 0.89 ✓ |
| ARIMA(0,1,2) | 175.3 | 181.5 | 0.89 ✓ |
| ARIMA(1,1,0) | 180.7 | 185.3 | 0.83 ✓ |
Winner: ARIMA(0,1,1): lowest AIC and BIC, residuals are white noise.
Estimated parameters: (drift), , .
Note how narrow the win is: ARIMA(1,1,1) and ARIMA(0,1,2) both nest ARIMA(0,1,1), and both come in exactly 2.0 AIC points behind. That is not a coincidence — see the trace below.
6-step forecast from :
The forecast jumps 5.6 on the first step even though the drift is only 1.96, because makes the model treat the last observation's fall (118 → 114) as a shock to be partly reversed. After that it grows by the drift.
The 95% prediction intervals are at and only at — they barely widen. For ARIMA(0,1,1) the -step variance is , and with the factor is nearly zero. A random walk's intervals grow like ; an MA(1) with near has almost cancelled the differencing, so its uncertainty is nearly horizon-independent.
ARIMA with d ≥ 1 has no drift term by default
In statsmodels, ARIMA(y, order=(0,1,1)) fits no constant when . On this series that model forecasts a flat 114.3 forever (below the last observation), and scores AIC 197.4 instead of 173.3. If your series has a visible trend, you must pass trend='t', or you will silently fit a driftless model and wonder why the forecast is flat.
Running the whole workflow in Python
import numpy as np
from statsmodels.tsa.stattools import adfuller
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.stats.diagnostic import acorr_ljungbox
import warnings
warnings.filterwarnings("ignore")
# Example series
y = np.array([50,51,54,55,60,56,64,63,65,64,63,67,
72,73,77,77,80,81,80,84,85,90,90,92,
96,99,99,101,104,109,106,106,119,115,118,114], dtype=float)
# Phase 1: determine d
print("ADF raw:", adfuller(y)[1].round(4)) # 0.9991 → non-stationary
print("ADF diff:", adfuller(np.diff(y))[1].round(4)) # 0.0023 → d=1
# Phase 2 + 3: fit candidates, compare AIC/BIC, check residuals
# trend='t' keeps the drift; without it, d>=1 fits no constant at all
for order in [(0,1,1), (1,1,1), (0,1,2), (1,1,0)]:
res = ARIMA(y, order=order, trend='t').fit()
lb = acorr_ljungbox(res.resid, lags=10, return_df=True)["lb_pvalue"].min()
print(f"ARIMA{order}: AIC={res.aic:.1f} BIC={res.bic:.1f} LjungBox_min_p={lb:.3f}")
# Phase 4: forecast
best = ARIMA(y, order=(0,1,1), trend='t').fit()
forecast = best.get_forecast(steps=6)
print("\n6-step forecast:")
print(forecast.summary_frame(alpha=0.05)[["mean","mean_ci_lower","mean_ci_upper"]])
Step through the selection
ARIMA order selection — the AIC grid, and how little it wins by
36 monthly observations, rising from 50 to 114. Before fitting anything, the identification phase has to answer one question: is this stationary? The autocorrelations of the raw series answer it — 0.92, 0.83, 0.75, 0.65 — decaying slowly and staying positive for many lags rather than dying out. That slow decay is the signature of a **unit root**: the series has no level it returns to, so its mean depends on where you start looking.
Box-Jenkins end to end on 36 observations: the raw ACF's slow decay identifies a unit root, the variance of the differenced series picks d by a visible U-shape, the correlogram of Δy proposes MA(1), and a 9-model AIC grid confirms it. Then two measured payoffs. Over-differencing raises the variance 3x and pushes θ̂ toward the −1 boundary — the unit MA root the extra difference itself created. And the winner's AIC margin over its nesting rival is 1.96, which is not a near-miss but a ceiling: when the extra parameter is useless the entire difference is the penalty constant, so AIC can never prefer the simpler nested model by more than 2. BIC's ln T = 3.56 makes the same call twice as decisively.
What to notice:
- Frame 2: choosing without a hypothesis test. The variance of is 436.6, 11.9, 35.2, 120.4 for . The minimum is the answer, and the U-shape shows why over-differencing is a real cost rather than a stylistic preference.
- Frame 7: the over-differencing payoff. Differencing twice triples the variance and pushes from to . That is not coincidence: differencing an already-stationary series applies an MA filter with a unit root , so the estimator must spend a parameter cancelling a root the analyst introduced, right on the boundary of the parameter space where estimation behaves worst.
- Frame 8: why the margin is 2.00. Every model that nests ARIMA(0,1,1) comes in at , , AIC — very close to for redundant parameters. When the extra coefficient is useless the log-likelihood does not move, so the entire gap is the penalty term. AIC can never prefer the simpler nested model by more than 2 per parameter, which is why a 2-point AIC win is weak evidence and BIC's penalty is the sharper instrument.
The trace estimates by conditional sum of squares rather than exact MLE, so its absolute AIC values sit about 0.9 below statsmodels'. The ranking is identical across all nine candidates.
Related concepts
- ACF & PACF: Reading the Correlogram: step 1b: reading the plots to identify p and q
- ARIMA Models: the full lesson with AR/MA model definitions
- Walk-Forward Validation: evaluating forecast accuracy without data leakage