Interventions & Potential OutcomesOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Interventions & Potential Outcomes

The previous lesson argued that causation differs from correlation. This one gives the tools to quantify a causal effect: the potential-outcomes framework (what would have happened?), the do-operator and backdoor criterion (when can observational data answer a causal question?), and the estimators that put numbers on "did it work?" This is a causal upgrade to the loop's evaluation slot: instead of measuring predictive accuracy, you estimate the effect of an intervention — the question A/B tests and deployment feedback actually care about.

Real-world examples: estimating a drug's effect from a trial, measuring an A/B test's lift, evaluating a policy or job-training program, and adjusting observational data to approximate an experiment.

Potential outcomes and counterfactuals

For a binary treatment T∈{0,1}T \in \{0, 1\}, each unit has two potential outcomes: Y(1)Y(1) if treated and Y(0)Y(0) if not. The individual causal effect is Yi(1)−Yi(0)Y_i(1) - Y_i(0).

The fundamental problem of causal inference

For any individual you only ever observe one potential outcome — the one for the treatment they actually got. The other is a counterfactual: forever unobserved. You can never measure an individual causal effect directly; the whole field is about recovering averages despite this missing-data problem.

So we target population averages. The Average Treatment Effect:

ATE=E[Y(1)−Y(0)]\text{ATE} = \mathbb{E}[Y(1) - Y(0)]

Why the naive difference is biased

The obvious estimate (average outcome of the treated minus the untreated) equals the ATE only if the groups are comparable. With confounding they aren't:

E[Y∣T=1]−E[Y∣T=0]⏟naive (observed)=ATE+confounding bias⏟groups differ\underbrace{\mathbb{E}[Y\mid T=1] - \mathbb{E}[Y\mid T=0]}_{\text{naive (observed)}} = \text{ATE} + \underbrace{\text{confounding bias}}_{\text{groups differ}}

The bias is the difference between the groups' baseline outcomes. Randomization makes it zero (groups equal in expectation); observational data needs adjustment.

The do-operator and interventional distributions

Pearl's do-operator formalizes intervention: P(Y∣do(T=1))P(Y \mid \text{do}(T=1)) is the distribution of YY if we reach in and set T=1T=1 for everyone — as opposed to P(Y∣T=1)P(Y\mid T=1), which is what we see among those who happened to be treated. Graphically, do(T)\text{do}(T) deletes all arrows into TT (including from confounders), which is exactly what randomization does physically. The ATE is E[Y∣do(1)]−E[Y∣do(0)]\mathbb{E}[Y\mid \text{do}(1)] - \mathbb{E}[Y\mid \text{do}(0)].

The backdoor criterion and adjustment

When can observational data identify a causal effect? The backdoor criterion: if a set of variables ZZ blocks every non-causal ("backdoor") path from TT to YY (intuitively, if ZZ contains all the confounders), then you can recover the effect by adjusting for ZZ:

P(Y∣do(T))=∑zP(Y∣T,Z=z) P(Z=z)P(Y \mid \text{do}(T)) = \sum_{z} P(Y \mid T, Z=z)\,P(Z=z)

In words: estimate the effect within each stratum of the confounders, then average over the confounder distribution. This is the adjustment (backdoor) formula — it turns a causal query into ordinary conditional probabilities you can estimate, given you measured the right ZZ.

Three assumptions make it work

Adjustment recovers the ATE under: (1) unconfoundedness — you measured and adjusted for all confounders (untestable, the big one); (2) positivity — every unit has a non-zero chance of each treatment (so each stratum has both groups); (3) consistency — the observed outcome equals the potential outcome for the treatment received. Violate (1) and your estimate is biased no matter how much data you have.

Estimating the effect in practice

  • Stratification / regression adjustment: the backdoor formula directly; include confounders as covariates.
  • Propensity scores: model P(T∣Z)P(T\mid Z) (probability of treatment given confounders); match or weight (IPW) treated and control units with similar propensity to mimic a randomized comparison.
  • Doubly-robust estimators: combine an outcome model and a propensity model; consistent if either is correct.
  • Randomize when you can: an RCT/A-B test makes ZZ irrelevant by design; adjustment is the fallback when experiments are impossible or unethical.

Estimating per-individual effects (who benefits most from treatment, the targeting question behind marketing and medicine) is uplift modeling, built exactly on this machinery:

Deep dive
Uplift Modeling & Causal Treatment Effects
Includes runnable notebook

Don't adjust for everything

Adjusting for confounders removes bias, but adjusting for a collider (a common effect of TT and YY) or a mediator (a variable on the causal path) introduces bias. "Control for more variables" is not safe; the causal graph tells you which ZZ is valid — the path-blocking machinery is d-separation. This is the practical payoff of drawing the structure first.

The cost of adjusting for the wrong thing

Controlling for more is not controlling for the right things

ZconfounderXtreatmentYoutcomeMdescendant of XCcollider
true direct effect0.9000
estimate1.6502
bias+0.7502
nothing1.6502+0.7502
Z — the confounder0.9078+0.0078
M — a descendant1.6162+0.7162
C — a collider0.0301-0.8699
Z and C0.0684-0.8316
Z and M0.8955-0.0045

Adjusting for Z alone recovers 0.9078 against a truth of 0.9. Then add the collider — one more variable you happened to measure — and the estimate collapses to 0.0684, worse than adjusting for nothing. Conditioning on a collider opens a path instead of closing one. No amount of staring at the data distinguishes the two cases; only the graph does.

A structural causal model with a known direct effect β = 0.9, sampled 4000 times. Boxed nodes are the ones you are conditioning on. The estimate is the ordinary-least-squares coefficient on X with those controls, solved in closed form — no fitting loop, no tuning.

A structural model with a known direct effect of β=0.9\beta = 0.9, one confounder ZZ, one descendant MM, and one collider CC. 4000 samples, and the estimate is just the OLS coefficient on XX with whichever controls you switch on:

adjust forβ^\hat\betabias
nothing1.6502+0.7502confounded
ZZ0.9078+0.0078correct backdoor set
MM (descendant)1.6162+0.7162does not close the back door
CC (collider)0.0301−0.8699effect destroyed
ZZ and CC0.0684−0.8316a correct answer, ruined
ZZ and MM0.8955−0.0045harmless here

Read the second row, then the fifth. Adjusting for ZZ alone lands on 0.90780.9078 against a truth of 0.90.9 — the backdoor criterion delivering exactly what it promises. Then add one more variable you happened to have in the table, and the estimate falls to 0.06840.0684: worse than adjusting for nothing at all. Conditioning on a collider does not close a path, it opens one, because learning CC makes XX and YY informative about each other through their shared effect.

The uncomfortable part is that nothing in the data distinguishes these rows. All six regressions run without complaint, all six produce a tidy coefficient and a small standard error, and the only thing that tells you which one answers your question is the graph you were willing to draw beforehand. "Throw every covariate in and let regularisation sort it out" is not a conservative choice. It is a causal assumption, made silently.

Exercise

The 'fundamental problem of causal inference' is that for any individual you can observe only one potential outcome. What is the standard response?

Need a hint?

If individual effects are unobservable, what do we target instead?

Exercise

How does P(Y | do(T=1)) differ from P(Y | T=1), and what does the do-operator do to the causal graph?

Need a hint?

Intervening vs observing; what happens to arrows into T?

Exercise

The backdoor adjustment formula lets you estimate a causal effect from observational data — but only under which key (untestable) assumption?

Need a hint?

What must the adjustment set Z contain?

Common mistakes

  1. Treating P(Y∣T)P(Y\mid T) as P(Y∣do(T))P(Y\mid \text{do}(T)). Observing the treated is not intervening; they differ exactly by confounding.
  2. Adjusting for colliders/mediators. It opens spurious paths or removes real effect — only adjust for confounders the graph identifies.
  3. Ignoring positivity. If some stratum has only treated (or only control) units, the effect there isn't identified and adjustment extrapolates blindly.
  4. Believing unconfoundedness because the model fits. It's an untestable assumption; a great predictive fit says nothing about hidden confounders.

Key takeaways

  • Each unit has potential outcomes Y(1),Y(0)Y(1), Y(0) but you observe only one — the fundamental problem; so we estimate the ATE =E[Y(1)−Y(0)]=\mathbb{E}[Y(1)-Y(0)].
  • The naive treated-minus-untreated difference equals the ATE plus confounding bias; randomization zeroes the bias, observational data must adjust.
  • The do-operator defines intervention (delete arrows into TT); the backdoor criterion + adjustment formula identify the effect if you measured all confounders.
  • Practical estimators (regression adjustment, propensity scores/IPW, doubly-robust) rely on unconfoundedness, positivity, consistency; don't adjust for colliders or mediators.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelatedGo deeper
Interventions & Potential OutcomesProbability & StatisticsCorrelation, Causation & …Bayesian NetworksBayesian InferenceHypothesis Testing & p-va…Uplift Modeling & Causal …