Calibration & UncertaintyOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Calibration & Uncertainty

A model that outputs probabilities is making a promise: "I'm 90% confident." Calibration asks whether that promise holds — among all the times it says 90%, is it actually right 90% of the time? A model can be highly accurate yet badly calibrated, and in high-stakes settings (medicine, autonomous systems, anything with a human in the loop) miscalibrated confidence is dangerous. This lesson covers how to measure calibration, how to fix it, and the two fundamentally different kinds of uncertainty a model faces.

The forecasting thread

Calibration is where the site's running daily-demand forecast (from the project-loop hub) finally gets judged honestly. A point forecast has no confidence to check, but the GP version produces a predictive interval, and a well-calibrated 90% band should contain the actual demand 90% of the time. Quantile loss and empirical coverage are exactly the calibration tools of this lesson, applied to a forecast instead of a classifier.

Real-world examples: a diagnostic model whose "85% malignant" must mean what it says; selective prediction (abstain when unsure); out-of-distribution detection; deciding when to escalate an LLM answer to a human.

Confidence ≠ correctness

Modern neural networks (especially deep ones with softmax outputs) are typically overconfident: they report 99% on predictions that are right far less often. Accuracy measures whether the top prediction is correct; calibration measures whether the probability attached to it is trustworthy. They're independent: you can be accurate and overconfident, or mediocre but well-calibrated.

Where miscalibration comes from

A model's stated job is to minimize a loss on a finite training set — recovering the true P(Y∣X)P(Y\mid X) is only a byproduct, guaranteed only under conditions real training never fully meets. The gap between predicted confidence and true probability has several distinct sources:

  • The loss barely aims at calibration, and over-optimizing it hurts. Proper scoring rules (cross-entropy, Brier) are minimized in expectation exactly at the true P(Y∣X)P(Y\mid X), so log loss genuinely "wants" to be calibrated. But (a) margin losses like the SVM hinge don't estimate probabilities at all — the output is a score, not a probability, which is why SVMs need post-hoc scaling; and (b) modern nets drive training loss toward zero long past the calibrated point, pushing logits to extremes → systematic overconfidence.
  • Model-class mismatch. If the true P(Y∣X)P(Y\mid X) isn't representable by your model family, the best fit is a biased projection onto it — miscalibrated wherever the family can't bend to match. This is the approximation-error term of the bias–variance decomposition.
  • The training data distribution: several separate effects: finite-sample noise in the estimates; class imbalance / reweighting, which makes the model learn P(Y∣X)P(Y\mid X) under the training prior rather than the deployment base rate; and label noise, which pushes confident predictions onto inherently ambiguous inputs.
  • Distribution shift (train ≠ deploy). The biggest real-world cause. Calibration is not preserved when the input distribution moves — a model calibrated on validation data drifts the moment deployment differs.
  • Irreducible noise the labels never reveal. Where the true probability is genuinely 0.70.7 (overlapping classes), hard 0/10/1 labels give the model no signal to stop at 0.70.7. This is the aleatoric floor (below), distinct from epistemic confident-extrapolation into regions with no data.

Regularization cuts across several of these: weight decay and especially label smoothing deliberately trade likelihood fit for generalization, moving the optimum away from the raw-calibrated point — sometimes improving calibration (label smoothing softens overconfidence), sometimes not. The upshot: because the causes live in the training process, the standard fixes are post-hoc — they leave the model alone and re-map its scores onto the diagonal using held-out data.

SourceMechanismFixable with more data?
Margin loss (SVM hinge)output is a score, not a probabilityNo: needs post-hoc scaling
Over-optimizing a proper losslogits driven to extremes → overconfidenceNo: inherent to the training dynamics
Regularization / label smoothingtrades likelihood fit for generalizationNo: a deliberate bias
Model-class mismatchtrue P(Y∣X)P(Y\mid X) not representableNo: approximation error
Finite-sample estimation errornoisy parameter estimatesYes (epistemic)
Class imbalance / reweightinglearns the training prior, not the deploy base rateYes: with base-rate correction
Label noisehard labels on ambiguous inputsPartly
Distribution shift (train ≠ deploy)calibration set ≠ deployment distributionNo: must recalibrate on target
Aleatoric noisetrue probability is intrinsically non-0/10/1No: irreducible
Epistemic ignoranceconfident extrapolation off the dataYes

Measuring calibration

Reliability diagram. Bin predictions by their confidence, and for each bin plot the actual accuracy against the average confidence. Perfect calibration lies on the diagonal (confidence = accuracy); bars below it mean overconfidence. The viz shows this, plus the effect of the fix below:

Calibration: reliability diagram & temperature scaling

confidenceaccuracy0.50.50.750.7511
avg confidence0.827
avg accuracy0.710
ECE0.120

Each bar groups predictions by confidence; its height is the actual accuracy in that bin. Perfect calibration lies on the diagonal (confidence = accuracy). The model is overconfident at temperature 1 (bars below the line). Raise the temperature to soften the probabilities; the Expected Calibration Error (ECE) bottoms out when the bars meet the diagonal.

Expected Calibration Error (ECE). Summarize the diagram in one number — the average gap between confidence and accuracy, weighted by how many predictions fall in each bin:

ECE=∑b=1BnbN ∣ acc(b)−conf(b) ∣\text{ECE} = \sum_{b=1}^{B} \frac{n_b}{N}\,\big|\,\text{acc}(b) - \text{conf}(b)\,\big|

Lower is better; 00 is perfectly calibrated. (ECE has known flaws — it depends on the binning — but it's the standard scalar.)

Fixing calibration: temperature scaling

The simplest and most effective fix is temperature scaling: divide the logits by a single scalar TT before the softmax, tuned on a validation set:

pi=softmax ⁣(zT)ip_i = \text{softmax}\!\left(\frac{\mathbf{z}}{T}\right)_i
  • T=1T = 1: unchanged.
  • T>1T > 1: softens the distribution → lower confidence (fixes overconfidence).
  • T<1T < 1: sharpens → higher confidence.

Because TT is a single parameter applied to all logits, it rescales confidence without changing the ranking, so accuracy is untouched while calibration improves. It's a cheap post-hoc step: train normally, then fit TT on held-out data. (Platt scaling and isotonic regression are alternatives.)

Temperature scaling can't change which class wins

Dividing all logits by the same TT is monotonic, so the argmax (the predicted class) is identical for every input. That's why it improves calibration for free: top-1 accuracy is exactly preserved, only the confidence numbers move. (This is the same temperature knob used in decoding; here it's tuned for calibration, not sampling.)

Temperature scaling is the right tool when one global knob suffices. When the miscalibration isn't a uniform stretch (you need a per-score correction rather than a single scalar), the two standard alternatives are Platt scaling (fit a logistic sigmoid on the scores) and isotonic regression (fit a monotonic step function, non-parametrically). The deep dive derives both, including the Pool-Adjacent-Violators algorithm and when each beats the other:

Deep dive
Platt Scaling and Isotonic Regression
Includes runnable notebook

Two kinds of uncertainty

Not all uncertainty is the same, and the distinction drives what you can do about it:

AleatoricEpistemic
Sourceinherent noise in the datathe model's ignorance
Exampleoverlapping classes, label noise, a blurry imagea region with no training data
Reducible?No: irreducible, even with infinite dataYes: shrinks with more data
Detect bythe predictive distribution's spreaddisagreement across models / Bayesian posterior

Aleatoric uncertainty is the noise floor — the irreducible noise variance σ2\sigma^2 that appears in the bias-variance decomposition, present even with infinite data. Epistemic uncertainty is what a Gaussian process or Bayesian model reports as it moves away from data, and it's why ensembles are useful: models trained from different seeds agree where they're confident and disagree where they're uncertain.

Estimating uncertainty in deep nets

  • Deep ensembles: train several models; the spread of their predictions estimates epistemic uncertainty. Simple and strong.
  • MC dropout: keep dropout on at inference and average many stochastic forward passes — a cheap approximation to a Bayesian posterior.
  • Bayesian methods (GPs, Bayesian NNs): model the posterior directly (see the Bayesian Methods course).

Exercise

A model is 95% accurate but, among predictions it makes with 99% confidence, it's correct only 80% of the time. What does this indicate?

Need a hint?

Is the problem accuracy, or the trustworthiness of the probability?

Exercise

Temperature scaling divides all logits by a single tuned scalar T before the softmax. Why does it improve calibration without changing accuracy?

Need a hint?

Does dividing every logit by the same T change which class has the largest logit?

Exercise

Which scenario is dominated by EPISTEMIC (rather than aleatoric) uncertainty?

Need a hint?

Which kind shrinks if you collect more data?

Common mistakes

  1. Trusting raw softmax probabilities. Deep nets are usually overconfident; calibrate before treating outputs as probabilities.
  2. Confusing accuracy with calibration. A 95%-accurate model can still be badly miscalibrated; measure ECE separately.
  3. Calibrating on the training set. Fit temperature/Platt scaling on held-out validation data, or you'll just memorize.
  4. Conflating aleatoric and epistemic uncertainty. More data fixes only the epistemic part; the aleatoric noise floor remains.

Key takeaways

  • Calibration asks whether predicted confidence matches empirical accuracy; it's independent of accuracy, and deep nets are typically overconfident.
  • Measure it with a reliability diagram and summarize with ECE (mean confidence–accuracy gap).
  • Temperature scaling (divide logits by a tuned TT) is a cheap post-hoc fix that improves calibration without changing accuracy.
  • Aleatoric uncertainty (data noise) is irreducible; epistemic uncertainty (model ignorance) shrinks with data — estimate it with ensembles, MC dropout, or Bayesian methods.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelatedGo deeper
Calibration & UncertaintyLinear & Logistic Regress…KNN & Decision TreesNeural NetworksEvaluating AI SystemsGaussian ProcessesLearning Theory & General…Classification & Regressi…Rank Ordering & Model Com…Ad Ranking & CTR Predicti…The ML Project LoopBias-Variance Decompositi…Platt Scaling and Isotoni…Concordance: the c-statis…Decile Analysis: Gains, L…Dropout: Inverted Dropout…