Uplift Modeling & Causal Treatment Effects
Standard recommendation predicts who will convert. Uplift modeling predicts who will convert because of an intervention — a discount, an email, a product recommendation. The target variable is the Individual Treatment Effect (ITE):
where is the outcome if user is treated and is the outcome if not. The fundamental problem: you can only observe one potential outcome per user. Uplift models estimate from A/B experiment data where treatment is randomly assigned.
The four uplift segments
Users fall into four segments based on their behavior with and without treatment:
| Segment | No treatment | Treatment | Uplift |
|---|---|---|---|
| Persuadables | Would not convert | Would convert | ✅ Positive |
| Sure things | Would convert | Would convert | 0 |
| Lost causes | Would not convert | Would not convert | 0 |
| Sleeping dogs | Would convert | Would not convert | ❌ Negative |
The goal is to target Persuadables and avoid Sleeping dogs — users who would convert anyway (Sure things) waste budget; Sleeping dogs are made worse by the intervention (e.g., a discount that insults a loyal customer).
S-Learner (Single Model)
Train a single model that takes both features and treatment indicator as inputs:
Estimate ITE by computing the model's prediction with and without treatment:
Pros: Simple, uses all data jointly, no need for two separate models.
Cons: If the treatment feature has weak main effects but strong interaction effects, regularized models may shrink its coefficient toward zero, underestimating uplift.
T-Learner (Two Models)
Train separate models for treated and control groups:
Pros: Each model is optimized for its own group, captures group-specific patterns.
Cons: High variance when treatment/control group sizes are imbalanced; the two models may have different implicit regularization that introduces bias into .
X-Learner (Cross-fitted)
Proposed by Künzel et al. (2019). Three-stage approach:
Stage 1: Train on control, on treated (same as T-Learner).
Stage 2: Compute imputed treatment effects using cross-group predictions:
Stage 3: Train two uplift models:
- : regress on features for treated units.
- : regress on features for control units.
Combine using propensity score :
Pros: Better performance when treatment/control sizes are very imbalanced (X-Learner upweights the larger group's estimate).
Cons: Requires propensity score estimation; 3-stage pipeline is complex.
Evaluation: Qini curve and AUUC
Standard accuracy metrics are meaningless for uplift models — the true ITE is never observed. Use ranking-based metrics:
Qini curve: Sort users by predicted uplift (descending). Plot cumulative incremental gains vs. fraction of population treated. A perfect model concentrates all Persuadables at the top.
AUUC (Area Under the Uplift Curve): normalize the Qini curve area. Higher AUUC = better uplift model.
Real-world applications from case studies
| Company | Use case | Estimated uplift |
|---|---|---|
| Netflix | Personalize "push notification to re-engage" targeting | +8% conversion vs. random targeting |
| Lyft | Incentive targeting (discount only high-uplift riders) | −12% promotion cost, flat revenue |
| Gojek | Targeted special offers for dormant users | +15% reactivation vs. blanket discount |
| Instacart | Budget-constrained promotion allocation | +9% ROAS |
| NYT | Paywall presentation personalization | +6% subscription conversion |
Causal forests
For non-parametric uplift estimation, Causal Forests (Wager & Athey, 2018) build an ensemble of causal trees that partition the feature space to find heterogeneous treatment effects:
The causalml and EconML Python packages implement causal forests with confidence intervals, enabling statistical significance testing of uplift estimates.
Related concepts
- Interventions & Potential Outcomes: potential outcomes and the ITE this models
- Correlation, Causation & Confounding: why randomized treatment assignment is what makes uplift estimable
- Bagging & Random Forests: the random-forest machinery causal forests build on