Evaluating Clusters & Choosing kOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Evaluating Clusters & Choosing k

Classification has accuracy. Regression has RMSE. Clustering has… a problem: there are no labels to check against, and K-Means will happily carve any dataset (including pure noise) into exactly kk pieces. Evaluating clusters means asking two questions: are these clusters real? and how many should there be? This is the evaluation slot in its hardest form — with no ground truth, you need internal validity measures (silhouette, the elbow) instead of accuracy.

Inertia and the elbow

K-Means' own objective (within-cluster sum of squares) always decreases as kk grows. More centroids, shorter distances, until k=nk = n gives inertia 0. So you can't pick the kk that minimizes it.

The elbow method plots inertia against kk and looks for the bend: the point where adding another cluster stops paying. Before the elbow, each new centroid claims a genuine group; after it, new centroids just split real groups in half for marginal gains.

The elbow is quick but subjective — many real curves bend gradually with no obvious knee. Treat it as a first look, not a verdict.

Silhouette: cohesion vs. separation

The silhouette coefficient scores each point by comparing its own cluster to the nearest other one:

s(i)=b(i)−a(i)max⁡(a(i), b(i))s(i) = \frac{b(i) - a(i)}{\max(a(i),\, b(i))}
  • a(i)a(i) — mean distance from point ii to others in its own cluster (cohesion)
  • b(i)b(i) — mean distance to points in the nearest other cluster (separation)

Reading the value:

s(i)s(i)Meaning
≈ +1comfortably inside its cluster, far from the next one
≈ 0sitting on the boundary between two clusters
< 0closer to a neighboring cluster: likely misassigned

Worked example

Five points in 1-D: cluster A = , cluster B = . For point i=2i = 2 (in cluster A):

  • a(2)a(2): mean distance to other A members = ∣2−1∣+∣2−3∣2=1+12=1.0\frac{|2-1| + |2-3|}{2} = \frac{1+1}{2} = 1.0
  • b(2)b(2): mean distance to B members = ∣2−8∣+∣2−9∣2=6+72=6.5\frac{|2-8| + |2-9|}{2} = \frac{6+7}{2} = 6.5 (nearest other cluster)
  • s(2)=6.5−1.0max⁡(1.0, 6.5)=5.56.5≈0.85s(2) = \frac{6.5 - 1.0}{\max(1.0,\, 6.5)} = \frac{5.5}{6.5} \approx 0.85 → well inside its cluster ✓

Average ss over all points and you get one number per kk — pick the kk that maximizes it. Even better, look at the silhouette plot (per-cluster profiles): a healthy clustering shows clusters of similar width, all above the average line; a bad one shows thin slivers and negative tails.

Metrics inherit assumptions

Silhouette and inertia both reward compact, round, well-separated blobs — the same geometry K-Means assumes. The two interleaved crescents that DBSCAN finds perfectly can score worse on silhouette than a wrong K-Means split of the same data. A metric can only validate clusters against its own definition of "good."

See it on real blobs — the data below has 3 true clusters. Move k and watch the bars: only the right k gives tall, even silhouettes in every cluster:

Silhouette analysis: is k right?

feature x₁feature x₂00.51
k2
mean silhouette0.469

Left: 3 true blobs clustered at your chosen k. Right: each point's silhouette s = (b−a)/max(a,b), grouped by cluster. At k=3 the bars are tall and even. At k=2 a merged cluster's points sit far from their own center (low s); at k>3 a split blob's points are nearly as close to the twin cluster as their own (s near 0).

Stability: the practitioner's check

A complementary test that assumes nothing about shape: resample and re-cluster. Take bootstrap samples (or random 80% subsets), run the algorithm, and measure how often pairs of points land together (e.g. with the adjusted Rand index between runs). Real structure survives perturbation; artifacts of one particular sample don't. If k=4k=4 produces nearly identical partitions across resamples while k=6k=6 scrambles, trust 4.

And when ground-truth labels do exist (evaluating against known categories), use adjusted Rand index or normalized mutual information — both compare partitions while correcting for chance agreement.

Exercise

A point has silhouette score s = −0.3. What does that tell you?

Need a hint?

Negative means b(i) < a(i) — compare the definitions.

Exercise

What does the 'elbow method' look for when choosing K in K-Means?

Need a hint?

Plot inertia vs K and look for a change in slope.

Key takeaways

  • Inertia always falls with kk — look for the elbow, but don't expect it to be sharp
  • Silhouette balances cohesion against separation: +1 great, 0 boundary, negative misassigned
  • Every internal metric carries shape assumptions; silhouette favors round blobs just like K-Means
  • Stability under resampling is the strongest label-free evidence that clusters are real

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelated
Evaluating Clusters & Choosing kLinear & Logistic Regress…K-Means ClusteringHierarchical Clustering &…Statistical Inference & C…