Interventions & Potential Outcomes
The previous lesson argued that causation differs from correlation. This one gives the tools to quantify a causal effect: the potential-outcomes framework (what would have happened?), the do-operator and backdoor criterion (when can observational data answer a causal question?), and the estimators that put numbers on "did it work?" This is a causal upgrade to the loop's evaluation slot: instead of measuring predictive accuracy, you estimate the effect of an intervention — the question A/B tests and deployment feedback actually care about.
Real-world examples: estimating a drug's effect from a trial, measuring an A/B test's lift, evaluating a policy or job-training program, and adjusting observational data to approximate an experiment.
Potential outcomes and counterfactuals
For a binary treatment , each unit has two potential outcomes: if treated and if not. The individual causal effect is .
The fundamental problem of causal inference
For any individual you only ever observe one potential outcome — the one for the treatment they actually got. The other is a counterfactual: forever unobserved. You can never measure an individual causal effect directly; the whole field is about recovering averages despite this missing-data problem.
So we target population averages. The Average Treatment Effect:
Why the naive difference is biased
The obvious estimate (average outcome of the treated minus the untreated) equals the ATE only if the groups are comparable. With confounding they aren't:
The bias is the difference between the groups' baseline outcomes. Randomization makes it zero (groups equal in expectation); observational data needs adjustment.
The do-operator and interventional distributions
Pearl's do-operator formalizes intervention: is the distribution of if we reach in and set for everyone — as opposed to , which is what we see among those who happened to be treated. Graphically, deletes all arrows into (including from confounders), which is exactly what randomization does physically. The ATE is .
The backdoor criterion and adjustment
When can observational data identify a causal effect? The backdoor criterion: if a set of variables blocks every non-causal ("backdoor") path from to (intuitively, if contains all the confounders), then you can recover the effect by adjusting for :
In words: estimate the effect within each stratum of the confounders, then average over the confounder distribution. This is the adjustment (backdoor) formula — it turns a causal query into ordinary conditional probabilities you can estimate, given you measured the right .
Three assumptions make it work
Adjustment recovers the ATE under: (1) unconfoundedness — you measured and adjusted for all confounders (untestable, the big one); (2) positivity — every unit has a non-zero chance of each treatment (so each stratum has both groups); (3) consistency — the observed outcome equals the potential outcome for the treatment received. Violate (1) and your estimate is biased no matter how much data you have.
Estimating the effect in practice
- Stratification / regression adjustment: the backdoor formula directly; include confounders as covariates.
- Propensity scores: model (probability of treatment given confounders); match or weight (IPW) treated and control units with similar propensity to mimic a randomized comparison.
- Doubly-robust estimators: combine an outcome model and a propensity model; consistent if either is correct.
- Randomize when you can: an RCT/A-B test makes irrelevant by design; adjustment is the fallback when experiments are impossible or unethical.
Estimating per-individual effects (who benefits most from treatment, the targeting question behind marketing and medicine) is uplift modeling, built exactly on this machinery:
Don't adjust for everything
Adjusting for confounders removes bias, but adjusting for a collider (a common effect of and ) or a mediator (a variable on the causal path) introduces bias. "Control for more variables" is not safe; the causal graph tells you which is valid — the path-blocking machinery is d-separation. This is the practical payoff of drawing the structure first.
The cost of adjusting for the wrong thing
Controlling for more is not controlling for the right things
| nothing | 1.6502 | +0.7502 |
| Z — the confounder | 0.9078 | +0.0078 |
| M — a descendant | 1.6162 | +0.7162 |
| C — a collider | 0.0301 | -0.8699 |
| Z and C | 0.0684 | -0.8316 |
| Z and M | 0.8955 | -0.0045 |
Adjusting for Z alone recovers 0.9078 against a truth of 0.9. Then add the collider — one more variable you happened to measure — and the estimate collapses to 0.0684, worse than adjusting for nothing. Conditioning on a collider opens a path instead of closing one. No amount of staring at the data distinguishes the two cases; only the graph does.
A structural causal model with a known direct effect β = 0.9, sampled 4000 times. Boxed nodes are the ones you are conditioning on. The estimate is the ordinary-least-squares coefficient on X with those controls, solved in closed form — no fitting loop, no tuning.
A structural model with a known direct effect of , one confounder , one descendant , and one collider . 4000 samples, and the estimate is just the OLS coefficient on with whichever controls you switch on:
| adjust for | bias | ||
|---|---|---|---|
| nothing | 1.6502 | +0.7502 | confounded |
| 0.9078 | +0.0078 | correct backdoor set | |
| (descendant) | 1.6162 | +0.7162 | does not close the back door |
| (collider) | 0.0301 | −0.8699 | effect destroyed |
| and | 0.0684 | −0.8316 | a correct answer, ruined |
| and | 0.8955 | −0.0045 | harmless here |
Read the second row, then the fifth. Adjusting for alone lands on against a truth of — the backdoor criterion delivering exactly what it promises. Then add one more variable you happened to have in the table, and the estimate falls to : worse than adjusting for nothing at all. Conditioning on a collider does not close a path, it opens one, because learning makes and informative about each other through their shared effect.
The uncomfortable part is that nothing in the data distinguishes these rows. All six regressions run without complaint, all six produce a tidy coefficient and a small standard error, and the only thing that tells you which one answers your question is the graph you were willing to draw beforehand. "Throw every covariate in and let regularisation sort it out" is not a conservative choice. It is a causal assumption, made silently.
Exercise
The 'fundamental problem of causal inference' is that for any individual you can observe only one potential outcome. What is the standard response?
Need a hint?
If individual effects are unobservable, what do we target instead?
Exercise
How does P(Y | do(T=1)) differ from P(Y | T=1), and what does the do-operator do to the causal graph?
Need a hint?
Intervening vs observing; what happens to arrows into T?
Exercise
The backdoor adjustment formula lets you estimate a causal effect from observational data — but only under which key (untestable) assumption?
Need a hint?
What must the adjustment set Z contain?
Common mistakes
- Treating as . Observing the treated is not intervening; they differ exactly by confounding.
- Adjusting for colliders/mediators. It opens spurious paths or removes real effect — only adjust for confounders the graph identifies.
- Ignoring positivity. If some stratum has only treated (or only control) units, the effect there isn't identified and adjustment extrapolates blindly.
- Believing unconfoundedness because the model fits. It's an untestable assumption; a great predictive fit says nothing about hidden confounders.
Key takeaways
- Each unit has potential outcomes but you observe only one — the fundamental problem; so we estimate the ATE .
- The naive treated-minus-untreated difference equals the ATE plus confounding bias; randomization zeroes the bias, observational data must adjust.
- The do-operator defines intervention (delete arrows into ); the backdoor criterion + adjustment formula identify the effect if you measured all confounders.
- Practical estimators (regression adjustment, propensity scores/IPW, doubly-robust) rely on unconfoundedness, positivity, consistency; don't adjust for colliders or mediators.
Related concepts
- Correlation, Causation & Confounding: the intuition this lesson formalizes
- Bayesian Inference: the conditional-probability machinery behind the adjustment formula
- Hypothesis Testing & p-values: how the estimated effect's significance is judged