t-SNE & UMAPOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

t-SNE & UMAP

PCA finds linear projections. But what if the data lies on a curved manifold? t-SNE and UMAP are non-linear methods that preserve local structure — nearby points stay nearby in the low-dimensional embedding.

Real-world examples: Visualizing single-cell RNA data (thousands of cell types), exploring word embeddings, visualizing deep network activations, cluster discovery in high-dimensional data.

t-SNE (t-distributed Stochastic Neighbor Embedding)

The idea

  1. Compute pairwise similarities in high-dimensional space using Gaussian kernels
  2. Initialize random points in 2D
  3. Compute pairwise similarities in 2D using Student-t distribution (heavier tails)
  4. Minimize the KL divergence between the two similarity distributions
KL(P∥Q)=∑i≠jpijlog⁡pijqij\text{KL}(P \| Q) = \sum_{i \neq j} p_{ij} \log \frac{p_{ij}}{q_{ij}}

where pijp_{ij} = high-D similarities, qijq_{ij} = low-D similarities.

The actual similarities

High-D (Gaussian). Point ii "picks neighbor" jj with conditional probability shrinking with squared distance, bandwidth σi\sigma_i set per-point (see perplexity below):

pj∣i=exp⁡(−∥xi−xj∥2/2σi2)∑k≠iexp⁡(−∥xi−xk∥2/2σi2),pij=pj∣i+pi∣j2np_{j|i} = \frac{\exp(-\|\mathbf{x}_i-\mathbf{x}_j\|^2 / 2\sigma_i^2)}{\sum_{k\neq i}\exp(-\|\mathbf{x}_i-\mathbf{x}_k\|^2 / 2\sigma_i^2)}, \qquad p_{ij} = \frac{p_{j|i} + p_{i|j}}{2n}

The symmetrization makes pij=pjip_{ij}=p_{ji} and guarantees every point contributes (so outliers aren't ignored).

Low-D (Student-t, 1 d.o.f.). The heavy tail is the whole trick — it lets dissimilar points sit far apart cheaply:

qij=(1+∥yi−yj∥2)−1∑k≠l(1+∥yk−yl∥2)−1q_{ij} = \frac{(1+\|\mathbf{y}_i-\mathbf{y}_j\|^2)^{-1}}{\sum_{k\neq l}(1+\|\mathbf{y}_k-\mathbf{y}_l\|^2)^{-1}}

Minimizing the KL by gradient descent gives a clean force law — attract when p>qp>q, repel when q>pq>p:

∂ KL∂yi=4∑j(pij−qij)(yi−yj) (1+∥yi−yj∥2)−1\frac{\partial \,\text{KL}}{\partial \mathbf{y}_i} = 4\sum_{j}(p_{ij}-q_{ij})(\mathbf{y}_i-\mathbf{y}_j)\,(1+\|\mathbf{y}_i-\mathbf{y}_j\|^2)^{-1}

Why Student-t distribution in 2D?

In high dimensions, many points are far apart. In 2D, they'd be forced close together. The Student-t distribution has heavier tails, so distant points in high-D can remain distant in low-D without causing large gradients. This is the key innovation over earlier methods.

Perplexity = effective number of neighbors

Perplexity isn't an arbitrary knob — it's defined as 22 raised to the Shannon entropy of the conditional distribution PiP_i:

Perplexity(Pi)=2H(Pi),H(Pi)=−∑jpj∣ilog⁡2pj∣i.\text{Perplexity}(P_i) = 2^{H(P_i)}, \qquad H(P_i) = -\sum_{j} p_{j|i}\log_2 p_{j|i}.

For each point, t-SNE binary-searches σi\sigma_i until the perplexity hits your target. Interpretation: if PiP_i spreads evenly over mm neighbors, H=log⁡2mH=\log_2 m so perplexity =m=m — the effective neighbor count. Perplexity 30 means "tune each point's Gaussian so it has roughly 30 meaningful neighbors."

Worked mini-example. Suppose PiP_i puts probability evenly on exactly 4 neighbors (p=14p=\tfrac14 each, 0 elsewhere). Then H=−4⋅14log⁡214=log⁡24=2H = -4\cdot\tfrac14\log_2\tfrac14 = \log_2 4 = 2, so perplexity =22=4=2^2=4 — exactly the neighbor count. A peaked distribution like (0.97,0.01,0.01,0.01)(0.97, 0.01, 0.01, 0.01) has H≈0.24H\approx0.24, perplexity ≈1.18\approx1.18: effectively one neighbor.

  • Low perplexity (5-15): small σi\sigma_i, few effective neighbors → tight local clusters
  • High perplexity (30-100): large σi\sigma_i, many effective neighbors → broader global patterns
  • Typical default: 30

t-SNE Perplexity: Effect on Embedding

Class 1Class 2Class 3Class 4Class 5embedding dim 1 (arbitrary units)embedding dim 2
perplexity30
effective neighbors~30
structureClean clusters — the typical sweet spot

⚠ t-SNE cluster sizes and inter-cluster distances are NOT meaningful — only topology (which points are neighbors) matters. The same data produces drastically different-looking plots at different perplexity values.

Python
from sklearn.manifold import TSNE
from sklearn.datasets import load_digits
import matplotlib.pyplot as plt

digits = load_digits()
X, y = digits.data, digits.target

# t-SNE embedding
tsne = TSNE(n_components=2, perplexity=30, random_state=42, n_iter=1000)
X_tsne = tsne.fit_transform(X)

plt.figure(figsize=(10, 8))
scatter = plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=y, cmap='tab10', s=10, alpha=0.7)
plt.colorbar(scatter, label='Digit')
plt.title(f't-SNE (perplexity=30, KL={tsne.kl_divergence_:.3f})')
plt.show()

Limitations of t-SNE

  1. Slow: O(n2)O(n^2) in naive implementation (Barnes-Hut approximation: O(nlog⁡n)O(n \log n))
  2. Non-deterministic: Different runs give different embeddings
  3. Preserves local, not global structure: Cluster sizes and distances between clusters are not meaningful
  4. Cannot embed new points: No transform() method — must retrain

Don't over-interpret t-SNE plots

t-SNE cluster sizes, distances between clusters, and global geometry are NOT meaningful. Only the relative positions within clusters are reliable. A tight cluster means the points are similar; that's all you can conclude.

UMAP (Uniform Manifold Approximation and Projection)

UMAP is faster and often better at preserving global structure than t-SNE.

Advantages over t-SNE

Propertyt-SNEUMAP
SpeedO(nlog⁡n)O(n \log n) (Barnes–Hut)O(n)O(n): 10-100× faster
Global structurePoorBetter
Transform new dataNoYes
Scalability~10K points~100K+ points

Key parameters

  • n_neighbors: Local vs global structure (similar to perplexity)
    • Small (5-15): tight clusters, local structure
    • Large (50-200): broad patterns, global structure
  • min_dist: How tightly points pack together
    • Small (0.0-0.1): tight clusters
    • Large (0.5-1.0): spread out points
Python
import umap

reducer = umap.UMAP(n_neighbors=15, min_dist=0.1, random_state=42)
X_umap = reducer.fit_transform(X)

plt.figure(figsize=(10, 8))
scatter = plt.scatter(X_umap[:, 0], X_umap[:, 1], c=y, cmap='tab10', s=10, alpha=0.7)
plt.colorbar(scatter, label='Digit')
plt.title('UMAP (n_neighbors=15, min_dist=0.1)')
plt.show()

UMAP preserves more global structure

In t-SNE, a cluster of "8"s might appear closer to "3"s than to "6"s just because of local density. UMAP is more likely to preserve the actual relationships — 8s closer to 6s and 3s.

When to use what

GoalMethod
Quick visualization of clustersUMAP (faster)
Publication-quality plotst-SNE (more established)
Embedding new data pointsUMAP (has transform)
Large datasets (> 100K)UMAP
Preserving global structureUMAP
Very high-dimensional data (images, text)UMAP with PCA preprocessing

Recommended pipeline

Python
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

# For high-dimensional data, reduce with PCA first
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('pca', PCA(n_components=50)),      # reduce to 50D first
    ('umap', umap.UMAP(n_neighbors=15)) # then UMAP to 2D
])
X_embedded = pipeline.fit_transform(X_high_dim)

Exercise

In UMAP, what does increasing 'n_neighbors' do?

Need a hint?

Think about whether you're capturing local or global structure.

Exercise

In t-SNE, what does the perplexity parameter control?

Need a hint?

Think about the balance between local and global structure.

Exercise

Set the t-SNE perplexity for exploring cluster structure in a 3000-point dataset.

5102.50200
Need a hint?

Perplexity ≈ effective number of neighbors. Too low → fractured clusters; too high → washed-out global structure.

Exercise

When should you use UMAP instead of PCA for visualization?

Need a hint?

Think about linear vs non-linear relationships.

Common mistakes

  1. Using t-SNE/UMAP for clustering: These are visualization tools, not clustering algorithms. Use their output as features for K-Means or DBSCAN, but don't cluster directly on the embedding.
  2. Interpreting distances literally: Only local neighborhoods are meaningful. A cluster being "larger" or "farther" doesn't mean anything quantitative.
  3. Not preprocessing: Both methods are sensitive to feature scales. Always standardize first. For high-dimensional data, reduce with PCA first.
  4. Running on the full dataset: UMAP on 1M+ points is fine, but t-SNE may need subsampling.

Key takeaways

  • t-SNE and UMAP are non-linear methods for visualizing high-dim data in 2-D.
  • They preserve local neighborhoods; cluster sizes/distances aren't literal.
  • t-SNE's perplexity and UMAP's n_neighbors balance local vs global structure.
  • Denoise with PCA first; never feed these embeddings to a downstream model.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelated
t-SNE & UMAPClusteringLinear Algebra for MLProbability & StatisticsEntropy & KL DivergenceHierarchical Clustering &…Principal Component Analy…PCA in PracticeTopic Modeling with BERTo…