Anomaly & Outlier DetectionOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Anomaly & Outlier Detection

Most ML predicts the common case. Anomaly detection flips the goal: find the rare, the unexpected, the don't-look-like-the-rest. The anomalies are exactly the points you have almost no labeled examples of (fraud, intrusions, equipment failures, manufacturing defects), so the problem demands different tools and a different mindset than ordinary classification.

Real-world examples: credit-card fraud, network-intrusion detection, predictive maintenance (a failing machine), manufacturing QA, and out-of-distribution detection for safe ML deployment.

Why it's not just classification

Anomalies are rare and diverse: you might have a handful of fraud examples and millions of normal ones, and the next fraud may look nothing like the last. This breaks supervised classification two ways — there aren't enough positive labels, and the positive class isn't a coherent cluster. So most anomaly detection is unsupervised or semi-supervised: model what normal looks like, then flag whatever deviates.

anomaly score s(x) high  ⟺  x is far from the model of normal\text{anomaly score } s(\mathbf{x}) \text{ high} \;\Longleftrightarrow\; \mathbf{x}\text{ is far from the model of normal}

The output is a continuous anomaly score; a threshold turns it into a decision.

The families of methods

Statistical. Assume a distribution for normal data and flag low-probability points. The classic rule: flag points beyond μ±3σ\mu \pm 3\sigma (a z-score), or use a robust version (median/MAD) less swayed by the outliers themselves. Simple, interpretable, great for low-dimensional or per-feature checks.

Distance / density-based. An anomaly is far from its neighbors or in a sparse region:

  • kNN distance: score = distance to the kk-th nearest neighbor.
  • LOF (Local Outlier Factor): compares a point's local density to its neighbors', catching anomalies that are only locally unusual (a point normal globally but isolated within its cluster).
  • DBSCAN: points that join no cluster are labeled noise (anomalies).

Isolation Forest. A clever inversion: build random trees that split on random features at random thresholds; anomalies get isolated in fewer splits because they're far from the mass of data. The anomaly score is the average path length to isolate a point — short paths = anomalous. Fast, scalable, and a strong default.

Reconstruction-based. Train an autoencoder (or PCA) only on normal data; it learns to reconstruct normal inputs well. At test time, a high reconstruction error flags an anomaly — the model can't represent something it never learned. The deep-learning workhorse for high-dimensional data (images, sensor streams).

One-Class SVM. Learn a boundary enclosing the normal data; points outside are anomalies.

Choosing a method by data shape

Low-dimensional, roughly Gaussian → statistical. Clustered with varying density → LOF/DBSCAN. Tabular, mixed features, large → Isolation Forest. High-dimensional (images, signals) → autoencoder reconstruction. There's no universal best; match the method to the geometry of "normal."

The threshold problem

Every method produces a score; you must pick a cutoff. This is a precision/recall trade-off under extreme imbalance:

  • Too low → flood of false alarms (alert fatigue; the team ignores the system).
  • Too high → misses real anomalies (the costly failure you built this to catch).

Set the threshold from a target alert budget ("we can investigate 50/day") or a target recall on known past anomalies. Because anomalies are rare, accuracy is a useless metric (a "always normal" detector is 99.9% accurate) — evaluate with precision-recall AUC, recall at a fixed alert rate, or precision@k.

The cutoff you didn't know you'd chosen

Every threshold is a price you already agreed to

normalanomaliescost-optimal 2.22F1-optimal 3.31
recall0.6351
precision0.0459
alerts / day13,831
of them false95.4%
missed anomalies365
expected cost here49,684
best achievable49,684
cost of using the F1 threshold+50%

Set the ratio to 1000 and drag your threshold onto the yellow line: catching 88% of anomalies means accepting 101,057 alerts a day, 99.1% of them false. That is not a bad model — it is what 0.1% prevalence costs, and no cut escapes it. The fix is triage, enrichment or a better score, never a better threshold.

Normal traffic N(0,1) against anomalies N(2.6,1.1), evaluated analytically so the far tail is exact. Counts are per 1M events a day at the chosen prevalence. The teal line is your threshold; the yellow one is whatever the current cost ratio makes optimal.

"Precision/recall trade-off" understates it. Every threshold is a statement about what a miss is worth relative to a false alarm, and picking one by maximising F1 makes that statement silently. At 0.1% prevalence over 1M events a day:

cost(miss) ÷ cost(alarm)thresholdrecallprecisionalerts/day
13.930.11330.7275156
103.100.32470.25141,291
1002.220.63510.045913,831
10001.280.88490.0088101,057

The F1-optimal threshold is 3.313.31 — recall 0.25930.2593, precision 0.35750.3575, 725 alerts. It is a defensible operating point, and it is 2% off at a ratio of 10, 50% off at 100, and 244% off at 1000. F1 is not a neutral default; it encodes an assumption that a missed anomaly costs roughly ten false alarms. If fraud costs you £5,000 and an analyst's review costs £5, your ratio is 1000 and F1 is choosing a threshold nearly four times worse than the one you want.

And then the arithmetic imbalance forces. At ratio 1000, catching 88% of anomalies means accepting 101,057 alerts a day of which 99.1% are false. That is not a bad detector (the model is unchanged); it is what 0.1% prevalence costs, and no choice of cutoff escapes it. Drag prevalence up to 1% and the same recall becomes affordable. This is why the real fixes are triage, enrichment and a better score rather than a better threshold, and why "we can investigate 50/day" is a more honest starting point than any metric.

Contamination and drift

Two traps: (1) your "normal" training data is often contaminated with undetected anomalies, which the model then learns as normal — use robust methods or clean iteratively. (2) Normal drifts over time (seasonality, new user behavior), so a static threshold goes stale; monitor and re-fit, tying back to production monitoring.

Exercise

Why is anomaly detection usually framed as unsupervised (model 'normal' and flag deviations) rather than supervised classification?

Need a hint?

How many — and how varied — are the anomaly examples?

Exercise

Isolation Forest scores a point by the average number of random splits needed to isolate it. Why do anomalies get SHORT path lengths?

Need a hint?

Where do anomalies sit relative to the dense mass of normal data?

Exercise

A fraud detector flags 0.1% of transactions as anomalous. Why is classification accuracy a poor way to evaluate it?

Need a hint?

What accuracy does a detector that flags nothing achieve?

Common mistakes

  1. Using accuracy. With 0.1% anomalies, predicting "all normal" scores 99.9% — meaningless. Use PR-AUC / recall at a fixed alert rate.
  2. Assuming clean training data. Real "normal" sets contain hidden anomalies (contamination) that poison the model of normal.
  3. A static threshold forever. Normal behavior drifts; the cutoff must be monitored and updated.
  4. Forcing supervised classification. Too few and too varied positives — model normal and flag deviations instead.

Key takeaways

  • Anomaly detection finds rare, diverse deviations; with too few/varied positive labels it's usually unsupervised — model normal, flag the rest with a continuous anomaly score.
  • Method families: statistical (z-score), distance/density (kNN, LOF, DBSCAN), Isolation Forest (anomalies isolate fast), reconstruction (autoencoder error), one-class SVM — matched to the data's geometry.
  • A threshold turns scores into decisions, trading false alarms against misses; set it by an alert budget or target recall.
  • Accuracy is misleading under extreme imbalance — use PR-AUC / recall@alert-rate; watch for contamination and drift.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelated
Anomaly & Outlier DetectionLinear & Logistic Regress…Neural NetworksModel Evaluation & Valida…Monitoring & ObservabilityHierarchical Clustering &…AutoencodersFraud Detection at Scale