Time Series Fundamentals
A time series is any sequence of observations ordered by time. Stock prices sampled every minute, hourly temperature readings at a weather station, millisecond ECG signals from a heart monitor — all share the same structure: the order of observations carries information that cannot be shuffled away. That single fact reshapes the data stage of the ML project loop: you can't shuffle-and-split, features are built from the past, and evaluation must respect time. This course is also the home of the site's running demand-forecasting thread.
Formally, we write a time series as , where is a discrete time index and is the total number of observations. The key difference from an ordinary dataset is that successive values are not independent: knowing tells you something about that you would not know from an arbitrary earlier observation.
This dependence is both the challenge and the opportunity of time series analysis. It means standard i.i.d. assumptions break down, but it also means the past genuinely predicts the future.
Components of a time series
Real-world time series are rarely pure noise. They carry structure that can be separated into four additive (or multiplicative) components:
- Trend (): the long-run direction of the series — upward, downward, or flat. Slow growth in global average temperature. Rising e-commerce revenue year over year.
- Seasonality (): regular, fixed-period oscillations. Retail sales spike every December. Power demand peaks at 6 pm on weekdays. The period is known in advance.
- Cyclic (): oscillations with variable period, typically spanning multiple years. Business cycles, housing booms and busts. Unlike seasonality, the timing and amplitude are not fixed.
- Irregular/Residual (): what is left after the structured components are removed — unexplained noise and one-off shocks.
Two decomposition models capture these components:
Additive model — appropriate when the seasonal amplitude stays roughly constant regardless of the trend level:
Multiplicative model — appropriate when the seasonal amplitude grows proportionally with the trend (common in economic series):
Taking of a multiplicative model converts it to additive: . This is the main practical reason to log-transform a series before fitting an additive model.
Time Series Decomposition
Decompose a 48-month series into trend, seasonality, and residual. Switch between additive (y = T + S + R) and multiplicative (y = T × S × R) models.
Stationarity
A time series is weakly stationary (also called covariance-stationary) if three conditions hold simultaneously:
- — the mean is constant over time
- — the variance is constant over time
- — the autocovariance at lag depends only on , not on where in the series you are
Intuitively, a stationary series looks the same at any point in time: no trending mean, no widening spread, no shifting seasonal amplitude.
Why does it matter? Almost every classical forecasting model (ARIMA, exponential smoothing, state-space models) assumes stationarity. Parameter estimates from a non-stationary series are unreliable: the mean computed on the first half of the series will differ from the mean computed on the second half, so any model trained on one period will misfire on another.
The Augmented Dickey-Fuller (ADF) test
Eyeballing a plot is not sufficient. The ADF test provides a formal statistical test for stationarity.
The null hypothesis is that the series has a unit root — a specific form of non-stationarity where the series follows a random walk. A p-value below 0.05 lets you reject the null and conclude the series is stationary (at least for the unit-root type of non-stationarity).
ADF test in Python
from statsmodels.tsa.stattools import adfuller
result = adfuller(y, autolag="AIC")
print(f"ADF Statistic: {result[0]:.4f}")
print(f"p-value: {result[1]:.4f}")
print(f"Lags used: {result[2]}")
# p < 0.05 → reject null → series is stationary
# p > 0.05 → fail to reject → evidence of non-stationarity
ADF is not omniscient
The ADF test is designed to detect unit-root non-stationarity. It will not catch structural breaks (a sudden level shift in 2008) or heteroskedasticity (variance changing over time). Always complement the test with a visual inspection.
Making a series stationary
Two complementary tools handle the most common sources of non-stationarity.
Differencing
The first difference removes a linear trend:
If (a deterministic linear trend), then , which has a constant mean .
The second difference removes a quadratic trend. In ARIMA notation, counts the number of differences applied. Most real series need or .
Seasonal differencing: — removes a seasonal pattern of period (e.g. for monthly data with yearly seasonality).
Log transformation
When variance grows proportionally with the level of the series (typical in financial and economic data), a log transform stabilises it:
The two transforms combine naturally: apply log first to stabilise variance, then difference to remove trend.
Worked example. Suppose — a series growing at exactly 10% per period. Taking logs: . The first differences of are all approximately — a constant series with mean 0.0953 and near-zero variance. The growth rate has been linearised and stabilised in a single step.
Autocorrelation
The autocorrelation at lag measures the linear relationship between and :
Note always. For a stationary series, (symmetry), so we only plot non-negative lags.
Two plots are central to time series analysis:
- ACF (Autocorrelation Function): plots versus lag . Captures all correlation between and , including the indirect paths through intermediate lags.
- PACF (Partial ACF): plots the correlation between and after removing the linear influence of all lags between them. Analogous to partial regression coefficients.
Both plots include significance bands at (the 95% confidence interval under the white-noise null hypothesis). Spikes outside these bands are considered statistically significant.
The ACF and PACF plots are the primary diagnostic tools for identifying AR and MA orders. The ACF measures raw correlation at each lag; the PACF measures the correlation after removing the effect of all shorter lags — analogous to partial regression coefficients. Together they reveal the "fingerprint" of the underlying data-generating process.
Python implementation
import pandas as pd
import numpy as np
from statsmodels.tsa.stattools import adfuller
from statsmodels.tsa.seasonal import seasonal_decompose
import matplotlib.pyplot as plt
# Load airline passengers (classic benchmark dataset)
from statsmodels.datasets import get_rdataset
airline = get_rdataset("AirPassengers").data
y = airline["value"].values
dates = pd.date_range("1949-01", periods=len(y), freq="MS")
# ── Step 1: visual inspection ─────────────────────────────────────────────────
plt.figure(figsize=(12, 4))
plt.plot(dates, y)
plt.title("Monthly airline passengers (1949–1960)")
plt.ylabel("Passengers (thousands)")
plt.tight_layout()
plt.show()
# ── Step 2: test for stationarity ────────────────────────────────────────────
result = adfuller(y, autolag="AIC")
print(f"ADF Statistic: {result[0]:.4f}")
print(f"p-value: {result[1]:.4f}") # >> 0.05 → non-stationary (trend + seasonality)
# ── Step 3: make stationary ───────────────────────────────────────────────────
log_y = np.log(y) # stabilise variance
diff_log_y = np.diff(log_y) # remove trend
result2 = adfuller(diff_log_y, autolag="AIC")
print(f"\nAfter log + first difference:")
print(f"ADF p-value: {result2[1]:.4f}") # < 0.05 → stationary
# ── Step 4: decompose the original series ─────────────────────────────────────
decomp = seasonal_decompose(y, model="multiplicative", period=12)
fig, axes = plt.subplots(4, 1, figsize=(12, 8), sharex=True)
axes[0].plot(dates, decomp.observed, label="Observed"); axes[0].set_ylabel("Observed")
axes[1].plot(dates, decomp.trend, label="Trend"); axes[1].set_ylabel("Trend")
axes[2].plot(dates, decomp.seasonal, label="Seasonal"); axes[2].set_ylabel("Seasonal")
axes[3].plot(dates, decomp.resid, label="Residual"); axes[3].set_ylabel("Residual")
plt.suptitle("Multiplicative decomposition — airline passengers")
plt.tight_layout()
plt.show()
# ── Step 5: plot ACF and PACF ─────────────────────────────────────────────────
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
plot_acf(diff_log_y, lags=24, ax=ax1, title="ACF — log-differenced series")
plot_pacf(diff_log_y, lags=24, ax=ax2, title="PACF — log-differenced series")
plt.tight_layout()
plt.show()
Exercise
Monthly airline passenger data shows an upward trend and summer peaks that are much larger in later years than earlier years. Which decomposition model fits best?
Need a hint?
Does the seasonal amplitude stay constant or grow with the trend?
Exercise
A time series has an upward trend AND variance that increases over time. Which sequence of transformations makes it stationary?
Need a hint?
One transformation stabilizes variance; another removes the trend.
Key takeaways
- A time series has four components — trend, seasonality, cycle, and residual. Decomposing them clarifies which structures are predictable.
- Stationarity (constant mean, variance, and autocovariance) is assumed by most classical models. The ADF test provides a formal check.
- Differencing removes trends; log-transforming stabilises variance. Apply log first, then difference.
- The ACF and PACF plots reveal how past values correlate with the present and are the primary tools for model identification.
Related concepts
- ARIMA Models: the next lesson builds directly on stationarity and the ACF/PACF patterns introduced here
- Recurrent Neural Networks: how deep learning approaches the same sequential structure
- Linear Regression: the OLS intuition underpins AR model estimation