Calibration & Uncertainty
A model that outputs probabilities is making a promise: "I'm 90% confident." Calibration asks whether that promise holds — among all the times it says 90%, is it actually right 90% of the time? A model can be highly accurate yet badly calibrated, and in high-stakes settings (medicine, autonomous systems, anything with a human in the loop) miscalibrated confidence is dangerous. This lesson covers how to measure calibration, how to fix it, and the two fundamentally different kinds of uncertainty a model faces.
The forecasting thread
Calibration is where the site's running daily-demand forecast (from the project-loop hub) finally gets judged honestly. A point forecast has no confidence to check, but the GP version produces a predictive interval, and a well-calibrated 90% band should contain the actual demand 90% of the time. Quantile loss and empirical coverage are exactly the calibration tools of this lesson, applied to a forecast instead of a classifier.
Real-world examples: a diagnostic model whose "85% malignant" must mean what it says; selective prediction (abstain when unsure); out-of-distribution detection; deciding when to escalate an LLM answer to a human.
Confidence ≠ correctness
Modern neural networks (especially deep ones with softmax outputs) are typically overconfident: they report 99% on predictions that are right far less often. Accuracy measures whether the top prediction is correct; calibration measures whether the probability attached to it is trustworthy. They're independent: you can be accurate and overconfident, or mediocre but well-calibrated.
Where miscalibration comes from
A model's stated job is to minimize a loss on a finite training set — recovering the true is only a byproduct, guaranteed only under conditions real training never fully meets. The gap between predicted confidence and true probability has several distinct sources:
- The loss barely aims at calibration, and over-optimizing it hurts. Proper scoring rules (cross-entropy, Brier) are minimized in expectation exactly at the true , so log loss genuinely "wants" to be calibrated. But (a) margin losses like the SVM hinge don't estimate probabilities at all — the output is a score, not a probability, which is why SVMs need post-hoc scaling; and (b) modern nets drive training loss toward zero long past the calibrated point, pushing logits to extremes → systematic overconfidence.
- Model-class mismatch. If the true isn't representable by your model family, the best fit is a biased projection onto it — miscalibrated wherever the family can't bend to match. This is the approximation-error term of the bias–variance decomposition.
- The training data distribution: several separate effects: finite-sample noise in the estimates; class imbalance / reweighting, which makes the model learn under the training prior rather than the deployment base rate; and label noise, which pushes confident predictions onto inherently ambiguous inputs.
- Distribution shift (train ≠ deploy). The biggest real-world cause. Calibration is not preserved when the input distribution moves — a model calibrated on validation data drifts the moment deployment differs.
- Irreducible noise the labels never reveal. Where the true probability is genuinely (overlapping classes), hard labels give the model no signal to stop at . This is the aleatoric floor (below), distinct from epistemic confident-extrapolation into regions with no data.
Regularization cuts across several of these: weight decay and especially label smoothing deliberately trade likelihood fit for generalization, moving the optimum away from the raw-calibrated point — sometimes improving calibration (label smoothing softens overconfidence), sometimes not. The upshot: because the causes live in the training process, the standard fixes are post-hoc — they leave the model alone and re-map its scores onto the diagonal using held-out data.
| Source | Mechanism | Fixable with more data? |
|---|---|---|
| Margin loss (SVM hinge) | output is a score, not a probability | No: needs post-hoc scaling |
| Over-optimizing a proper loss | logits driven to extremes → overconfidence | No: inherent to the training dynamics |
| Regularization / label smoothing | trades likelihood fit for generalization | No: a deliberate bias |
| Model-class mismatch | true not representable | No: approximation error |
| Finite-sample estimation error | noisy parameter estimates | Yes (epistemic) |
| Class imbalance / reweighting | learns the training prior, not the deploy base rate | Yes: with base-rate correction |
| Label noise | hard labels on ambiguous inputs | Partly |
| Distribution shift (train ≠ deploy) | calibration set ≠ deployment distribution | No: must recalibrate on target |
| Aleatoric noise | true probability is intrinsically non- | No: irreducible |
| Epistemic ignorance | confident extrapolation off the data | Yes |
Measuring calibration
Reliability diagram. Bin predictions by their confidence, and for each bin plot the actual accuracy against the average confidence. Perfect calibration lies on the diagonal (confidence = accuracy); bars below it mean overconfidence. The viz shows this, plus the effect of the fix below:
Calibration: reliability diagram & temperature scaling
Each bar groups predictions by confidence; its height is the actual accuracy in that bin. Perfect calibration lies on the diagonal (confidence = accuracy). The model is overconfident at temperature 1 (bars below the line). Raise the temperature to soften the probabilities; the Expected Calibration Error (ECE) bottoms out when the bars meet the diagonal.
Expected Calibration Error (ECE). Summarize the diagram in one number — the average gap between confidence and accuracy, weighted by how many predictions fall in each bin:
Lower is better; is perfectly calibrated. (ECE has known flaws — it depends on the binning — but it's the standard scalar.)
Fixing calibration: temperature scaling
The simplest and most effective fix is temperature scaling: divide the logits by a single scalar before the softmax, tuned on a validation set:
- : unchanged.
- : softens the distribution → lower confidence (fixes overconfidence).
- : sharpens → higher confidence.
Because is a single parameter applied to all logits, it rescales confidence without changing the ranking, so accuracy is untouched while calibration improves. It's a cheap post-hoc step: train normally, then fit on held-out data. (Platt scaling and isotonic regression are alternatives.)
Temperature scaling can't change which class wins
Dividing all logits by the same is monotonic, so the argmax (the predicted class) is identical for every input. That's why it improves calibration for free: top-1 accuracy is exactly preserved, only the confidence numbers move. (This is the same temperature knob used in decoding; here it's tuned for calibration, not sampling.)
Temperature scaling is the right tool when one global knob suffices. When the miscalibration isn't a uniform stretch (you need a per-score correction rather than a single scalar), the two standard alternatives are Platt scaling (fit a logistic sigmoid on the scores) and isotonic regression (fit a monotonic step function, non-parametrically). The deep dive derives both, including the Pool-Adjacent-Violators algorithm and when each beats the other:
Two kinds of uncertainty
Not all uncertainty is the same, and the distinction drives what you can do about it:
| Aleatoric | Epistemic | |
|---|---|---|
| Source | inherent noise in the data | the model's ignorance |
| Example | overlapping classes, label noise, a blurry image | a region with no training data |
| Reducible? | No: irreducible, even with infinite data | Yes: shrinks with more data |
| Detect by | the predictive distribution's spread | disagreement across models / Bayesian posterior |
Aleatoric uncertainty is the noise floor — the irreducible noise variance that appears in the bias-variance decomposition, present even with infinite data. Epistemic uncertainty is what a Gaussian process or Bayesian model reports as it moves away from data, and it's why ensembles are useful: models trained from different seeds agree where they're confident and disagree where they're uncertain.
Estimating uncertainty in deep nets
- Deep ensembles: train several models; the spread of their predictions estimates epistemic uncertainty. Simple and strong.
- MC dropout: keep dropout on at inference and average many stochastic forward passes — a cheap approximation to a Bayesian posterior.
- Bayesian methods (GPs, Bayesian NNs): model the posterior directly (see the Bayesian Methods course).
Exercise
A model is 95% accurate but, among predictions it makes with 99% confidence, it's correct only 80% of the time. What does this indicate?
Need a hint?
Is the problem accuracy, or the trustworthiness of the probability?
Exercise
Temperature scaling divides all logits by a single tuned scalar T before the softmax. Why does it improve calibration without changing accuracy?
Need a hint?
Does dividing every logit by the same T change which class has the largest logit?
Exercise
Which scenario is dominated by EPISTEMIC (rather than aleatoric) uncertainty?
Need a hint?
Which kind shrinks if you collect more data?
Common mistakes
- Trusting raw softmax probabilities. Deep nets are usually overconfident; calibrate before treating outputs as probabilities.
- Confusing accuracy with calibration. A 95%-accurate model can still be badly miscalibrated; measure ECE separately.
- Calibrating on the training set. Fit temperature/Platt scaling on held-out validation data, or you'll just memorize.
- Conflating aleatoric and epistemic uncertainty. More data fixes only the epistemic part; the aleatoric noise floor remains.
Key takeaways
- Calibration asks whether predicted confidence matches empirical accuracy; it's independent of accuracy, and deep nets are typically overconfident.
- Measure it with a reliability diagram and summarize with ECE (mean confidence–accuracy gap).
- Temperature scaling (divide logits by a tuned ) is a cheap post-hoc fix that improves calibration without changing accuracy.
- Aleatoric uncertainty (data noise) is irreducible; epistemic uncertainty (model ignorance) shrinks with data — estimate it with ensembles, MC dropout, or Bayesian methods.
Related concepts
- Learning Theory & Generalization: the irreducible-noise term that is aleatoric uncertainty
- Gaussian Processes: a model whose epistemic uncertainty is calibrated by construction
- Classification Metrics: the accuracy-style metrics calibration complements