Minkowski Distance Family
Every algorithm that relies on "nearness" (KNN, K-Means, hierarchical clustering, kernel methods) needs a formula for distance. The Minkowski family unifies the most common metrics under a single parameter , letting you interpolate between Manhattan and Euclidean and beyond.
The four standard metrics
| Metric | Formula | Typical use |
|---|---|---|
| Euclidean | Default for continuous features | |
| Manhattan | High-dimensional, sparse, or grid-like data | |
| Minkowski () | Generalizes both; choose via CV | |
| Chebyshev | When the largest gap dominates |
The master Minkowski formula and its special cases
Special cases:
- : Manhattan — sum the absolute gaps; like walking a city grid, no diagonals.
- : Euclidean — square the gaps, sum, square-root; the ruler distance.
- : Chebyshev — only the single largest coordinate gap survives, since .
All the metrics on a worked example
Take to .
Per-coordinate absolute gaps:
Feed gaps into each formula:
Manhattan (): add the gaps.
Euclidean (): square, sum, square-root.
Minkowski (): cube the gaps, sum, take the cube root.
Chebyshev (): the maximum gap wins.
Ordering: . Larger lets the single biggest gap dominate while smaller gaps fade, so the distance decreases toward as .
How the parameter p changes the geometry
The unit ball (the set of all points at distance 1 from the origin) changes shape with :
| Unit ball shape | KNN neighborhood | |
|---|---|---|
| 1 | Diamond (square rotated 45°) | Axis-aligned diamond; gaps in each coordinate count separately |
| 2 | Circle / sphere | Isotropic; most natural for continuous data without strong axes |
| Square / hypercube | Square neighborhood; dominated by the single largest gap |
As increases from 1 to , the ball transitions from diamond to circle to square. Features with the largest individual gap gain more influence relative to the rest.
Feature scaling is critical
All Minkowski metrics treat coordinate magnitudes as meaningful. A feature measured in millimeters (range 0–10 000) completely overwhelms one in meters (range 0–10). Always normalize or standardize before computing distances.
Computing and comparing every metric in Python
import numpy as np
x = np.array([1, 2, 3], dtype=float)
y = np.array([4, 6, 3], dtype=float)
gaps = np.abs(x - y) # [3, 4, 0]
d1 = gaps.sum() # Manhattan: 7.0
d2 = np.sqrt((gaps**2).sum()) # Euclidean: 5.0
d3 = (gaps**3).sum() ** (1/3) # Minkowski p=3: ~4.497
dinf = gaps.max() # Chebyshev: 4.0
print(f"Manhattan (p=1): {d1:.4f}")
print(f"Euclidean (p=2): {d2:.4f}")
print(f"Minkowski (p=3): {d3:.4f}")
print(f"Chebyshev (p→∞): {dinf:.4f}")
# Verify limit: as p grows, Minkowski → Chebyshev
for p in [1, 2, 3, 5, 10, 50, 100]:
dp = (gaps**p).sum() ** (1/p)
print(f" p={p:3d}: d_p = {dp:.6f} (Chebyshev = {dinf:.6f})")
Categorical features: the Hamming distance
For features that are categories rather than numbers, Euclidean distance is meaningless. The Hamming distance counts the fraction of positions where the two vectors disagree:
A Hamming distance of 0 means the vectors are identical; a distance of 1 means they differ in every coordinate.
Related concepts
- K-Nearest Neighbors: the full lesson that uses these metrics and discusses when to choose each
- K-Means: Full Algorithm: another distance-based algorithm (always Euclidean)
- Hierarchical Clustering Linkage Methods: single/complete/average linkage all build on pairwise distances