Time Series FundamentalsOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Time Series Fundamentals

A time series is any sequence of observations ordered by time. Stock prices sampled every minute, hourly temperature readings at a weather station, millisecond ECG signals from a heart monitor — all share the same structure: the order of observations carries information that cannot be shuffled away. That single fact reshapes the data stage of the ML project loop: you can't shuffle-and-split, features are built from the past, and evaluation must respect time. This course is also the home of the site's running demand-forecasting thread.

Deep dive
The ML Project Loop
Includes runnable notebook

Formally, we write a time series as {yt}t=1T\{y_t\}_{t=1}^T, where tt is a discrete time index and TT is the total number of observations. The key difference from an ordinary dataset is that successive values are not independent: knowing y100y_{100} tells you something about y101y_{101} that you would not know from an arbitrary earlier observation.

This dependence is both the challenge and the opportunity of time series analysis. It means standard i.i.d. assumptions break down, but it also means the past genuinely predicts the future.

Components of a time series

Real-world time series are rarely pure noise. They carry structure that can be separated into four additive (or multiplicative) components:

  • Trend (TtT_t): the long-run direction of the series — upward, downward, or flat. Slow growth in global average temperature. Rising e-commerce revenue year over year.
  • Seasonality (StS_t): regular, fixed-period oscillations. Retail sales spike every December. Power demand peaks at 6 pm on weekdays. The period is known in advance.
  • Cyclic (CtC_t): oscillations with variable period, typically spanning multiple years. Business cycles, housing booms and busts. Unlike seasonality, the timing and amplitude are not fixed.
  • Irregular/Residual (RtR_t): what is left after the structured components are removed — unexplained noise and one-off shocks.

Two decomposition models capture these components:

Additive model — appropriate when the seasonal amplitude stays roughly constant regardless of the trend level:

yt=Tt+St+Rty_t = T_t + S_t + R_t

Multiplicative model — appropriate when the seasonal amplitude grows proportionally with the trend (common in economic series):

yt=Tt×St×Rty_t = T_t \times S_t \times R_t

Taking log⁡\log of a multiplicative model converts it to additive: log⁡yt=log⁡Tt+log⁡St+log⁡Rt\log y_t = \log T_t + \log S_t + \log R_t. This is the main practical reason to log-transform a series before fitting an additive model.

Time Series Decomposition

modelT + S + R
observations48 months
OriginalTrendSeasonalResidualm1m13m25m37m48

Decompose a 48-month series into trend, seasonality, and residual. Switch between additive (y = T + S + R) and multiplicative (y = T × S × R) models.

Stationarity

A time series is weakly stationary (also called covariance-stationary) if three conditions hold simultaneously:

  1. E[yt]=μ\mathbb{E}[y_t] = \mu — the mean is constant over time
  2. Var(yt)=σ2\text{Var}(y_t) = \sigma^2 — the variance is constant over time
  3. Cov(yt,yt+k)=γk\text{Cov}(y_t, y_{t+k}) = \gamma_k — the autocovariance at lag kk depends only on kk, not on where in the series you are

Intuitively, a stationary series looks the same at any point in time: no trending mean, no widening spread, no shifting seasonal amplitude.

Why does it matter? Almost every classical forecasting model (ARIMA, exponential smoothing, state-space models) assumes stationarity. Parameter estimates from a non-stationary series are unreliable: the mean computed on the first half of the series will differ from the mean computed on the second half, so any model trained on one period will misfire on another.

The Augmented Dickey-Fuller (ADF) test

Eyeballing a plot is not sufficient. The ADF test provides a formal statistical test for stationarity.

The null hypothesis is that the series has a unit root — a specific form of non-stationarity where the series follows a random walk. A p-value below 0.05 lets you reject the null and conclude the series is stationary (at least for the unit-root type of non-stationarity).

ADF test in Python

Python
from statsmodels.tsa.stattools import adfuller

result = adfuller(y, autolag="AIC")
print(f"ADF Statistic: {result[0]:.4f}")
print(f"p-value:       {result[1]:.4f}")
print(f"Lags used:     {result[2]}")
# p < 0.05 → reject null → series is stationary
# p > 0.05 → fail to reject → evidence of non-stationarity

ADF is not omniscient

The ADF test is designed to detect unit-root non-stationarity. It will not catch structural breaks (a sudden level shift in 2008) or heteroskedasticity (variance changing over time). Always complement the test with a visual inspection.

Making a series stationary

Two complementary tools handle the most common sources of non-stationarity.

Differencing

The first difference removes a linear trend:

Δyt=yt−yt−1\Delta y_t = y_t - y_{t-1}

If yt=α+βt+εty_t = \alpha + \beta t + \varepsilon_t (a deterministic linear trend), then Δyt=β+(εt−εt−1)\Delta y_t = \beta + (\varepsilon_t - \varepsilon_{t-1}), which has a constant mean β\beta.

The second difference Δ2yt=Δyt−Δyt−1\Delta^2 y_t = \Delta y_t - \Delta y_{t-1} removes a quadratic trend. In ARIMA notation, dd counts the number of differences applied. Most real series need d=1d = 1 or d=2d = 2.

Seasonal differencing: Δmyt=yt−yt−m\Delta_m y_t = y_t - y_{t-m} — removes a seasonal pattern of period mm (e.g. m=12m = 12 for monthly data with yearly seasonality).

Log transformation

When variance grows proportionally with the level of the series (typical in financial and economic data), a log transform stabilises it:

zt=log⁡(yt)z_t = \log(y_t)

The two transforms combine naturally: apply log first to stabilise variance, then difference to remove trend.

Worked example. Suppose y=[100,110,121,133.1]y = [100, 110, 121, 133.1] — a series growing at exactly 10% per period. Taking logs: log⁡(y)≈[4.605,4.700,4.796,4.891]\log(y) \approx [4.605, 4.700, 4.796, 4.891]. The first differences of log⁡(y)\log(y) are all approximately log⁡(1.1)≈0.0953\log(1.1) \approx 0.0953 — a constant series with mean 0.0953 and near-zero variance. The growth rate has been linearised and stabilised in a single step.

Autocorrelation

The autocorrelation at lag kk measures the linear relationship between yty_t and yt−ky_{t-k}:

ρk=Cov(yt,yt−k)Var(yt)\rho_k = \frac{\text{Cov}(y_t, y_{t-k})}{\text{Var}(y_t)}

Note ρ0=1\rho_0 = 1 always. For a stationary series, ρk=ρ−k\rho_k = \rho_{-k} (symmetry), so we only plot non-negative lags.

Two plots are central to time series analysis:

  • ACF (Autocorrelation Function): plots ρk\rho_k versus lag kk. Captures all correlation between yty_t and yt−ky_{t-k}, including the indirect paths through intermediate lags.
  • PACF (Partial ACF): plots the correlation between yty_t and yt−ky_{t-k} after removing the linear influence of all lags between them. Analogous to partial regression coefficients.

Both plots include significance bands at ±1.96/T\pm 1.96 / \sqrt{T} (the 95% confidence interval under the white-noise null hypothesis). Spikes outside these bands are considered statistically significant.

The ACF and PACF plots are the primary diagnostic tools for identifying AR and MA orders. The ACF measures raw correlation at each lag; the PACF measures the correlation after removing the effect of all shorter lags — analogous to partial regression coefficients. Together they reveal the "fingerprint" of the underlying data-generating process.

Deep dive
ACF & PACF: Reading the Correlogram
Includes runnable notebook

Python implementation

Python
import pandas as pd
import numpy as np
from statsmodels.tsa.stattools import adfuller
from statsmodels.tsa.seasonal import seasonal_decompose
import matplotlib.pyplot as plt

# Load airline passengers (classic benchmark dataset)
from statsmodels.datasets import get_rdataset
airline = get_rdataset("AirPassengers").data
y = airline["value"].values
dates = pd.date_range("1949-01", periods=len(y), freq="MS")

# ── Step 1: visual inspection ─────────────────────────────────────────────────
plt.figure(figsize=(12, 4))
plt.plot(dates, y)
plt.title("Monthly airline passengers (1949–1960)")
plt.ylabel("Passengers (thousands)")
plt.tight_layout()
plt.show()

# ── Step 2: test for stationarity ────────────────────────────────────────────
result = adfuller(y, autolag="AIC")
print(f"ADF Statistic: {result[0]:.4f}")
print(f"p-value: {result[1]:.4f}")   # >> 0.05 → non-stationary (trend + seasonality)

# ── Step 3: make stationary ───────────────────────────────────────────────────
log_y = np.log(y)                    # stabilise variance
diff_log_y = np.diff(log_y)          # remove trend

result2 = adfuller(diff_log_y, autolag="AIC")
print(f"\nAfter log + first difference:")
print(f"ADF p-value: {result2[1]:.4f}")  # < 0.05 → stationary

# ── Step 4: decompose the original series ─────────────────────────────────────
decomp = seasonal_decompose(y, model="multiplicative", period=12)

fig, axes = plt.subplots(4, 1, figsize=(12, 8), sharex=True)
axes[0].plot(dates, decomp.observed,  label="Observed");  axes[0].set_ylabel("Observed")
axes[1].plot(dates, decomp.trend,     label="Trend");     axes[1].set_ylabel("Trend")
axes[2].plot(dates, decomp.seasonal,  label="Seasonal");  axes[2].set_ylabel("Seasonal")
axes[3].plot(dates, decomp.resid,     label="Residual");  axes[3].set_ylabel("Residual")
plt.suptitle("Multiplicative decomposition — airline passengers")
plt.tight_layout()
plt.show()

# ── Step 5: plot ACF and PACF ─────────────────────────────────────────────────
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
plot_acf(diff_log_y,  lags=24, ax=ax1, title="ACF — log-differenced series")
plot_pacf(diff_log_y, lags=24, ax=ax2, title="PACF — log-differenced series")
plt.tight_layout()
plt.show()

Exercise

Monthly airline passenger data shows an upward trend and summer peaks that are much larger in later years than earlier years. Which decomposition model fits best?

Need a hint?

Does the seasonal amplitude stay constant or grow with the trend?

Exercise

A time series has an upward trend AND variance that increases over time. Which sequence of transformations makes it stationary?

Need a hint?

One transformation stabilizes variance; another removes the trend.

Key takeaways

  • A time series has four components — trend, seasonality, cycle, and residual. Decomposing them clarifies which structures are predictable.
  • Stationarity (constant mean, variance, and autocovariance) is assumed by most classical models. The ADF test provides a formal check.
  • Differencing removes trends; log-transforming stabilises variance. Apply log first, then difference.
  • The ACF and PACF plots reveal how past values correlate with the present and are the primary tools for model identification.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelatedGo deeper
Time Series FundamentalsRecurrent Neural NetworksLinear & Logistic Regress…ARIMA ModelsRecurrent Neural NetworksLinear RegressionFrom Batch to StreamingThe ML Project LoopACF & PACF: Reading the C…The Augmented Dickey-Full…