Correlation, Causation & ConfoundingOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Correlation, Causation & Confounding

Machine learning is extraordinary at finding correlations, but most decisions we actually care about are causal: will this treatment help this patient? will this feature change increase retention? A model can predict perfectly and still be useless (or dangerous) for these questions, because correlation and causation come apart in systematic, predictable ways. This lesson builds the intuition for why, which the next lesson formalizes. Confounding is fundamentally a data problem in the ML project loop: a perfect optimizer on a perfectly-fit model still gives the wrong answer to a causal question if the data-generating process hides a common cause. No amount of hypothesis-space or optimization work fixes a broken causal structure.

Deep dive
The ML Project Loop
Includes runnable notebook

Real-world examples: "does this drug work?" (vs. is it correlated with recovery), "did the marketing campaign cause sales?", deciding whether a model's feature is actionable, and avoiding policies that optimize a proxy while harming the real goal.

Prediction is not causation

A supervised model learns P(Y∣X)P(Y \mid X) — what YY tends to be when we observe XX. A causal question asks what YY would be if we intervened to set XX — written P(Y∣do(X))P(Y \mid \text{do}(X)). These are different:

P(Y∣X=x)    ≠    P(Y∣do(X=x))in generalP(Y \mid X = x) \;\;\neq\;\; P(Y \mid \text{do}(X = x)) \quad\text{in general}

Observing that hospital patients on a certain drug recover more often (correlation) does not mean giving the drug causes recovery — maybe doctors prescribe it to healthier patients. The model that predicts recovery from "is on drug" is accurate and causally wrong at the same time.

Confounding: the core problem

A confounder is a variable that influences both the supposed cause and the effect, creating a correlation that isn't causal:

Text
        confounder Z (e.g. patient health)
        ╱                          ╲
       ▼                            ▼
   treatment X  ───?───►  outcome Y

Health ZZ drives both who gets the treatment XX and who recovers YY, so XX and YY are correlated even if XX does nothing. The classic spurious examples — ice-cream sales correlate with drownings (confounder: hot weather), shoe size correlates with reading ability in children (confounder: age) — are all this structure.

You cannot detect confounding from the data alone

No amount of staring at XX and YY reveals whether a confounder is inflating their correlation — the data look identical whether the link is causal or confounded. You need either an experiment or assumptions about the causal structure (a graph). This is why causal inference needs more than statistics: it needs a model of how the data was generated.

Simpson's paradox

The most vivid demonstration: an association can reverse when you account for a confounder. A treatment can appear worse overall yet be better in every subgroup, because the subgroups had different base rates and different treatment assignment.

P(recover∣treated)<P(recover∣untreated)overall, yet the reverse within every subgroupP(\text{recover}\mid\text{treated}) < P(\text{recover}\mid\text{untreated}) \quad\text{overall, yet the reverse within every subgroup}

The aggregate and the stratified analyses disagree, and which one is correct depends on the causal structure — whether the grouping variable is a confounder (adjust for it) or a mediator (don't). Statistics alone can't tell you; causal reasoning can.

Does the dose help? Two answers from one dataset

doserecoverymildmoderatesevere
severitydoserecovery+0.90
sicker patients are given more of it
true effect+0.90
pooled fit (ignore severity)-0.421
within stratum (adjusted)+0.896

The pooled fit now says the dose is harmful, while all three stratum fits slope up. Same 180 patients, same +0.90 effect in each one — only the assignment rule changed.

180 patients in three severity strata. By construction the dose helps everyone by exactly +0.90 recovery points per unit, and severity independently costs 2.5 points. The slider controls only how strongly severity drives the dose — the confounding arrow — so every change you see comes from that one arrow, never from the treatment's real effect.

The generating process here is fixed and known: the dose helps every patient by exactly +0.90+0.90 recovery points per unit, and severity independently costs 2.52.5 points. The slider changes one thing only — how strongly severity decides who gets which dose. Three things are worth doing with it:

  1. Start at 0. Nothing is confounded yet, and the pooled fit reports +0.898+0.898 against a true +0.90+0.90. The naive analysis is not wrong in principle; it is wrong under a particular structure.
  2. Drag it up. Near 0.350.35 the pooled slope crosses zero, and past that the aggregate says the drug harms patients while all three stratum lines still slope up. The three within-stratum estimates never move — they sit at +0.896+0.896 the entire time. Nothing about the treatment changed; only who received it.
  3. Press "randomise the dose". The strata still differ in recovery, and the slider still reads whatever you left it at, but the pooled fit snaps back to the truth. Randomisation doesn't measure the confounder or adjust for it. It deletes the arrow that made adjustment necessary.

The third point is the one worth carrying: adjustment recovers the effect if you knew to adjust, and randomisation recovers it without you knowing anything. That asymmetry is why the RCT is the gold standard and not merely a convenience.

Why ML cares

  • Actionability. A feature can be highly predictive yet useless to act on (it's an effect, not a cause, or it's confounded). Deciding what to change is causal.
  • Robustness / distribution shift. Correlations that ride on a confounder break when the confounder's distribution shifts; causal relationships are more stable.
  • Fairness. Whether a model is "using" a protected attribute is fundamentally a causal question about paths, not a correlation.
  • Optimizing proxies. Pushing on a correlate of the goal (not a cause) can move the metric while the real objective stagnates or worsens.

How causation is actually established

  • Randomized controlled trial (RCT). Randomly assign the treatment. Randomization severs the arrow from any confounder into XX — treated and untreated groups are comparable in expectation, so the observed difference is the causal effect. This is the gold standard, and it's exactly why A/B tests randomize.
  • Adjustment from observational data. When you can't experiment, if you can identify and measure the confounders, you can statistically adjust for them (stratify / control) to recover the causal effect — under assumptions covered next lesson.

Exercise

Ice-cream sales and drowning deaths are strongly correlated. What is the confounder, and why can't you spot the confounding from the two variables alone?

Need a hint?

What third variable drives both? And do confounded vs causal data look different?

Exercise

A model accurately predicts recovery from 'patient is on drug X.' Why might this be useless for deciding whether to prescribe drug X?

Need a hint?

What's the difference between P(Y|X) and P(Y|do(X))?

Exercise

Simpson's paradox: a treatment appears worse overall but better within every subgroup. Which analysis is correct?

Need a hint?

Does statistics alone decide, or the causal role of the grouping variable?

Common mistakes

  1. Reading a predictive coefficient as a causal effect. A regression weight is not "the effect of changing XX" unless confounding is handled.
  2. Controlling for the wrong variable. Adjusting for a confounder removes bias; adjusting for a mediator or collider introduces it. The causal structure decides.
  3. Trusting aggregate associations. Simpson's paradox shows aggregates can reverse the truth; the right stratification depends on causal roles.
  4. Assuming more data fixes it. Confounding bias does not shrink with sample size — a billion confounded observations are still confounded.

Key takeaways

  • Prediction learns P(Y∣X)P(Y\mid X); causation concerns P(Y∣do(X))P(Y\mid \text{do}(X)) — observing vs. intervening — and they differ in general.
  • A confounder drives both treatment and outcome, creating non-causal correlation; you cannot detect it from X,YX,Y alone — you need experiments or structural assumptions.
  • Simpson's paradox: associations can reverse under stratification, and the correct analysis depends on the causal roles of variables.
  • Randomization (RCT/A-B test) severs confounding and is the gold standard; otherwise recover effects by adjusting for measured confounders.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelatedGo deeper
Correlation, Causation & Confound…Probability & StatisticsInterventions & Potential…Continual Learning & Test…Bayesian NetworksThe ML Project LoopUplift Modeling & Causal …