Demand Forecasting in Production
An ARIMA model trained on a single aggregated demand signal is a starting point — not a production system. DoorDash forecasts delivery demand per restaurant per hour at 30-minute resolution. Uber models ride demand per geohex cell per 5-minute interval. Instacart predicts grocery item demand across thousands of SKUs and hundreds of zip codes. Each faces the same fundamental challenges: hierarchical structure, sparse history at fine granularity, special events, and real-time latency requirements.
The forecasting thread
This is where the site's running daily-demand problem reaches the deployment-feedback slot of the ML project loop. The toy series had one regime shift; a real system has a new one every week (a store opens, a promo lands, a holiday hits). The model that was accurate at launch rots, so production forecasting is a loop: forecast → monitor error → detect drift → retrain on fresh data → redeploy. The same closing move you'll see in Streaming ML.
Real-world examples: DoorDash (order volume forecasting for driver supply planning), Lyft (market-level and driver-level demand), Uber (surge pricing and driver rebalancing), Instacart (inventory and delivery slot planning), Foodpanda (kitchen capacity allocation), Swiggy (driver availability matching), ODA (grocery inventory forecasting), Oda (last-mile logistics).
Hierarchical time series
Most business forecasting problems are hierarchical: total national demand is the sum of regional demands, which are sums of store-level demands, which are sums of SKU-level demands.
Total demand
├── Region A
│ ├── City 1
│ │ ├── Restaurant #101
│ │ └── Restaurant #102
│ └── City 2
└── Region B
A naive approach: fit independent models at each level. The problem: forecasts at different levels won't be coherent — summing city-level forecasts may not equal the regional-level forecast.
Reconciliation approaches
| Approach | Method | Best for |
|---|---|---|
| Top-down | Forecast aggregate; disaggregate using historical proportions | Stable series, sparse lower levels |
| Bottom-up | Forecast leaf nodes; sum up | Low-count series where aggregation loses info |
| Optimal reconciliation (MinT) | Jointly minimize forecast error across all levels using GLS | When you need coherence + accuracy |
MinT (Minimum Trace) casts reconciliation as a linear projection:
where is the summing matrix encoding the hierarchy, are base forecasts, and is the covariance matrix of forecast errors. The result is a coherent set of forecasts that minimize total mean squared error across all levels.
The formula is short; the reason you need it is not. Walk the pipeline below on a simulated five-store hierarchy — forecast every node independently, watch the numbers fail to add up, then apply each reconciliation method as the same projection with a different . The method control re-runs everything, and the accuracy table averages over 200 draws of the future rather than one test window, because on a single window these methods differ by less than the noise.
Hierarchical forecasting: making the numbers add up
Business forecasting is never one series
5 stores roll up into 2 regions and one national total — 8 series in all, but only 5 of them are free. Every aggregate is a fixed sum of the leaves, encoded in the summing matrix S: 8 rows, 5 columns, one row per node.
That constraint is not a modelling choice, it is arithmetic about the business. Any set of forecasts that violates it is describing a world that cannot happen — and that is precisely what independent models produce.
Click any node to follow it through the walkthrough. S is shown beside the tree; every row is one node written as a sum of stores.
structure
8 nodes, 5 degrees of freedom · S is 8 × 5.
Five stores, two regions and a national total, forecast independently and then reconciled. The hierarchy is simulated, a damped-trend smoother is fitted per node with its own grid-searched parameters, and bottom-up, top-down, OLS and MinT are applied as the same projection S·G·ŷ with different G. The accuracy table averages over 200 draws of the future rather than one test window, because on a single window the differences between these methods are smaller than the noise.
Handling special events
Time series models extrapolate from history, but holidays, promotions, and external events create demand spikes outside the historical distribution. Models that ignore them produce large, systematic errors.
Prophet's additive decomposition
Facebook Prophet decomposes demand as:
- : trend (piecewise linear or logistic growth)
- : seasonality (Fourier series with weekly and yearly periods)
- : holiday effects — user-specified calendar events with individually estimated impact coefficients
- : noise
The holiday component is the key: supply a list of dates (Thanksgiving, Super Bowl, local public holidays) and Prophet estimates a separate additive effect for each. At DoorDash, promotions and restaurant campaigns are treated as "holidays" and injected into the Prophet decomposition.
External regressors for planned events
For promotional events where magnitude is known in advance (e.g., a planned $5 discount starting Tuesday), add the promotion indicator as an exogenous regressor in ARIMAX or as a future covariate in TFT (Temporal Fusion Transformer). The model learns the typical promotion lift from history and applies it to the future event.
Anomaly robustification
Black swan events (COVID lockdowns, extreme weather) contaminate the training signal. Two defenses:
- Robust STL decomposition: use median-based trend extraction, which is resistant to outliers.
- Truncated training window: weight recent data more heavily (exponential decay) so ancient anomalies don't distort the fitted model.
ETA prediction: forecasting with uncertainty
ETA (Estimated Time of Arrival) prediction is a special-case demand forecast: the target is a continuous duration (minutes to delivery) rather than a volume. Calibrated uncertainty is critical — telling a customer "30 minutes" when the true distribution is 20–50 minutes damages trust.
Feature engineering for ETA
| Feature | Source | Lag |
|---|---|---|
| Distance (crow-flies) | Maps API | — |
| Historical travel time for route | Log store | 30-min bins |
| Current traffic index (segment-level) | Real-time traffic feed | < 1 min |
| Courier speed percentile | Courier feature store | 1-hour rolling |
| Order complexity (# items, prep time) | Order system | — |
| Time of day, day of week | Clock | — |
Swiggy uses gradient boosted trees over these features; their ETA model improves primarily by adding more fine-grained traffic features, not by model complexity.
Quantile regression for uncertainty
Instead of predicting the mean ETA, predict multiple quantiles (p10, p50, p90):
Optimizing this pinball loss at produces the 90th percentile ETA — a conservative bound shown to the customer. DoorDash shows a range ("arriving in 25–40 minutes") instead of a point estimate, using the p25 and p75 quantiles.
Ensemble forecasting
No single model wins on all horizons and series types. Production systems blend:
- Statistical models (ETS/ARIMA/Prophet): robust on short history, no GPU required, fast to retrain.
- Gradient boosting (LightGBM/XGBoost): powerful for feature-rich tabular forecasting, handles missing values.
- Deep learning (N-BEATS, TFT, PatchTST): best on long histories and cross-series patterns (global models trained on all SKUs simultaneously).
The ensemble is typically a stacking regressor: a secondary model (often Ridge regression) trained on the validation-set predictions of each base model learns optimal blending weights. Instacart's system uses a similar approach: ETS + LightGBM + a transformer are blended, with the transformer upweighted for SKUs with 6+ months of history.
Backtesting ≠ production evaluation
Backtesting with a fixed test window overfits your model selection to that window. Use expanding window validation (walk-forward): repeatedly train on all data up to time , forecast steps ahead, advance by one period, repeat. This simulates the actual rolling-retrain production cadence.
Production architecture
Raw data (orders, traffic, weather)
↓
Feature store (precomputed lag features)
↓
Multiple base models trained on rolling windows
↓
Stacking / blending layer
↓
Coherency reconciliation (if hierarchical)
↓
Forecast served to downstream systems:
├─ Driver dispatch & supply planning
├─ Kitchen / prep time scheduling
└─ Customer ETA display
Uber's forecasting platform (built on Apache Flink) produces horizon-5-minute predictions for thousands of geo-cells, updating every 5 minutes. The entire pipeline (from raw data to deployed forecast) runs on an automated retraining cadence with automated regression detection.
Exercise
You forecast demand independently at the city level and the national level. After the forecast, you sum city forecasts and find they don't equal the national forecast. What is this problem called, and how does MinT address it?
Need a hint?
The key word is 'coherent'.
Exercise
Why does backtesting with a single fixed holdout window overestimate a forecasting model's production performance?
Need a hint?
Think about how a model gets selected based on that window.
Exercise
DoorDash shows customers a delivery time range (e.g., '25–40 minutes') instead of a point estimate. Which modeling approach produces this range?
Need a hint?
Think about what it means to predict the p25 and p75 of a distribution.
Key takeaways
- Hierarchical forecasting requires coherent reconciliation across levels (MinT / bottom-up) so sub-forecasts sum to the aggregate.
- Special events must be modeled explicitly — Prophet's holiday component, ARIMAX regressors, or TFT future covariates; generic models produce large systematic errors on known events.
- ETA prediction benefits from quantile regression to produce calibrated uncertainty intervals rather than point estimates.
- Ensemble + stacking consistently outperforms any single model; the blend weights themselves should be learned, not hand-tuned.
Related concepts
- ARIMA Models: the statistical baseline that ensembles always include
- Deep Learning for Time Series: the TFT and NBEATS models used in production ensembles
- Feature Stores: how lag features and rolling aggregates are computed and served at low latency