Evaluating Clusters & Choosing k
Classification has accuracy. Regression has RMSE. Clustering has… a problem: there are no labels to check against, and K-Means will happily carve any dataset (including pure noise) into exactly pieces. Evaluating clusters means asking two questions: are these clusters real? and how many should there be? This is the evaluation slot in its hardest form — with no ground truth, you need internal validity measures (silhouette, the elbow) instead of accuracy.
Inertia and the elbow
K-Means' own objective (within-cluster sum of squares) always decreases as grows. More centroids, shorter distances, until gives inertia 0. So you can't pick the that minimizes it.
The elbow method plots inertia against and looks for the bend: the point where adding another cluster stops paying. Before the elbow, each new centroid claims a genuine group; after it, new centroids just split real groups in half for marginal gains.
The elbow is quick but subjective — many real curves bend gradually with no obvious knee. Treat it as a first look, not a verdict.
Silhouette: cohesion vs. separation
The silhouette coefficient scores each point by comparing its own cluster to the nearest other one:
- — mean distance from point to others in its own cluster (cohesion)
- — mean distance to points in the nearest other cluster (separation)
Reading the value:
| Meaning | |
|---|---|
| ≈ +1 | comfortably inside its cluster, far from the next one |
| ≈ 0 | sitting on the boundary between two clusters |
| < 0 | closer to a neighboring cluster: likely misassigned |
Worked example
Five points in 1-D: cluster A = , cluster B = . For point (in cluster A):
- : mean distance to other A members =
- : mean distance to B members = (nearest other cluster)
- → well inside its cluster ✓
Average over all points and you get one number per — pick the that maximizes it. Even better, look at the silhouette plot (per-cluster profiles): a healthy clustering shows clusters of similar width, all above the average line; a bad one shows thin slivers and negative tails.
Metrics inherit assumptions
Silhouette and inertia both reward compact, round, well-separated blobs — the same geometry K-Means assumes. The two interleaved crescents that DBSCAN finds perfectly can score worse on silhouette than a wrong K-Means split of the same data. A metric can only validate clusters against its own definition of "good."
See it on real blobs — the data below has 3 true clusters. Move k and watch the bars: only the right k gives tall, even silhouettes in every cluster:
Silhouette analysis: is k right?
Left: 3 true blobs clustered at your chosen k. Right: each point's silhouette s = (b−a)/max(a,b), grouped by cluster. At k=3 the bars are tall and even. At k=2 a merged cluster's points sit far from their own center (low s); at k>3 a split blob's points are nearly as close to the twin cluster as their own (s near 0).
Stability: the practitioner's check
A complementary test that assumes nothing about shape: resample and re-cluster. Take bootstrap samples (or random 80% subsets), run the algorithm, and measure how often pairs of points land together (e.g. with the adjusted Rand index between runs). Real structure survives perturbation; artifacts of one particular sample don't. If produces nearly identical partitions across resamples while scrambles, trust 4.
And when ground-truth labels do exist (evaluating against known categories), use adjusted Rand index or normalized mutual information — both compare partitions while correcting for chance agreement.
Exercise
A point has silhouette score s = −0.3. What does that tell you?
Need a hint?
Negative means b(i) < a(i) — compare the definitions.
Exercise
What does the 'elbow method' look for when choosing K in K-Means?
Need a hint?
Plot inertia vs K and look for a change in slope.
Key takeaways
- Inertia always falls with — look for the elbow, but don't expect it to be sharp
- Silhouette balances cohesion against separation: +1 great, 0 boundary, negative misassigned
- Every internal metric carries shape assumptions; silhouette favors round blobs just like K-Means
- Stability under resampling is the strongest label-free evidence that clusters are real
Related concepts
- K-Means Clustering: the algorithm whose these metrics choose
- Hierarchical & DBSCAN: the shapes silhouette undervalues
- Statistical Inference: the resampling logic behind the stability check