Diversity, Cold Start & ExplorationOpen in Colab
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Diversity, Cold Start & Exploration

Optimizing NDCG@10 can produce a recommender that is technically accurate but practically terrible: it recommends the same ten items to everyone, ignores new users and products, and traps users in an ever-narrowing taste bubble. This lesson covers the three structural problems every production recommender must address beyond raw accuracy.

Real-world examples: Spotify's "Discover Weekly" (diversity by design), Netflix's causal inference for exploration, Pinterest's cold-start for new Pins, LinkedIn's cold-start for new job listings, Airbnb's neighborhood for new property listings.

The cold start problem

Cold start occurs when a user or item has too few interactions to produce reliable embeddings. It comes in three flavors:

FlavorProblemExample
New userNo interaction historyA just-signed-up user on Spotify
New itemNo engagement data yetA freshly listed product on Etsy
New contextUnseen feature combinationA user in a new geographic market

Content-based fallback

The most practical cold-start remedy: represent items by their features (text description, category, price, image embedding) rather than interaction-derived embeddings. When interaction data is sparse, fall back to content similarity.

For a new item ii with features fi\mathbf{f}_i, project to the embedding space:

vicold=Wproj fi\mathbf{v}_i^{\text{cold}} = W_{\text{proj}}\,\mathbf{f}_i

The projection WprojW_{\text{proj}} is learned during training to align content features with interaction-trained embeddings. At Airbnb, new listings' content embeddings (images, description text, amenities) are used immediately on day 0; they converge to interaction-based embeddings over ~2 weeks as bookings accumulate.

Meta-learning for fast user adaptation

MAML (Model-Agnostic Meta-Learning) trains a base model such that a small number of gradient steps on a new user's first few interactions produces a high-quality personalized model. In recommendation:

θ∗=arg⁡min⁡θ∑uL(fθ−α∇L(fθ,Dusupport), Duquery)\theta^* = \arg\min_\theta \sum_u \mathcal{L}\left(f_{\theta - \alpha \nabla \mathcal{L}(f_\theta, \mathcal{D}_u^{\text{support}})},\, \mathcal{D}_u^{\text{query}}\right)

The inner loop updates for user uu on their first kk interactions; the outer loop optimizes the initialization θ\theta to be maximally "fast-adaptable". Effective for platforms where users are highly heterogeneous and the onboarding sequence is short.

Onboarding intent capture

Many platforms explicitly ask new users for preferences at signup: Spotify's "pick your favorite artists," Tinder's "swipe on these profiles." These explicit signals bootstrap the embedding even before any implicit interaction occurs. The key is to show items that maximally partition the latent preference space (items that are highly discriminative), not just the most popular ones.

Warm-up strategy at Netflix

Netflix uses a combination: new users see a set of titles chosen to elicit a preference signal (explicit rating request), and in parallel are assigned to a "taste cluster" of existing users whose early ratings matched the new user's profile. The cluster's collaborative signal is used while the new user's personal data accumulates.

Deep dive
Cold Start: Strategies & Algorithms
Includes runnable notebook

Diversity beyond accuracy

A ranker optimizing average relevance will recommend the best item over and over — serving only the dominant taste cluster and creating a filter bubble. Diversity metrics measure how varied a recommendation list is:

  • Intra-list diversity (ILD): average pairwise dissimilarity of recommended items. ILD=2K(K−1)∑i≠jd(i,j)\text{ILD} = \frac{2}{K(K-1)}\sum_{i \neq j} d(i,j).
  • Coverage: fraction of the item catalog that ever appears in recommendations across users.
  • Serendipity: relevance-weighted surprise — items that are both liked and unexpected.

Maximal Marginal Relevance (MMR)

MMR re-ranks a candidate list to balance relevance and novelty. At each step, select the item that maximizes:

MMR(i)=λ⋅ri−(1−λ)⋅max⁡j∈Ssim(i,j)\text{MMR}(i) = \lambda \cdot r_i - (1-\lambda) \cdot \max_{j \in S} \text{sim}(i, j)

where rir_i is the relevance score, SS is the set of already-selected items, and sim\text{sim} is embedding cosine similarity. λ∈[0,1]\lambda \in [0,1] trades off relevance vs diversity. A greedy selection of the top-KK items via MMR takes O(KN)O(KN) and is trivially parallelizable.

Spotify applies MMR-style diversification at the re-ranking stage: after retrieval and ranking, the list is passed through a diversity optimizer that ensures no two consecutive songs share the same artist, genre, or BPM bracket.

Determinantal Point Processes (DPP)

DPPs assign higher probability to subsets of items that are both high quality and mutually diverse:

P(S)∝det⁡(LS)P(S) \propto \det(L_S)

where Lij=ri1/2 sim(i,j) rj1/2L_{ij} = r_i^{1/2} \,\text{sim}(i,j)\, r_j^{1/2} (the kernel matrix). The determinant is large when items have high quality (rir_i large) and low mutual similarity (off-diagonal terms small). DPPs are mathematically elegant but expensive to sample from exactly; in practice, greedy approximations achieve > 90% of optimal diversity in O(K2N)O(K^2 N).

Exploration vs exploitation

Even with a well-trained model, there are items the model thinks aren't relevant but would actually be enjoyed, because there's no data to learn from. Exploration is the deliberate injection of uncertainty-resolved queries.

Multi-armed bandits recap

The bandit problem (treated in full in Exploration Strategies & Model-Based RL): KK arms with unknown reward distributions. At each step, pull an arm to get a reward and update beliefs. The fundamental tension: exploit the current best arm, or explore an uncertain arm to reduce uncertainty.

Upper Confidence Bound (UCB): Add an exploration bonus proportional to uncertainty:

at=arg⁡max⁡i[μ^i+cln⁡tni]a_t = \arg\max_i \left[\hat{\mu}_i + c\sqrt{\frac{\ln t}{n_i}}\right]

Items pulled fewer times (nin_i small) get larger bonuses, ensuring every item gets explored eventually. UCB is provably sublinear in regret for stationary distributions.

Thompson Sampling: Maintain a posterior distribution P(μi)P(\mu_i) over each arm's mean. At each step, sample μ~i∼P(μi)\tilde{\mu}_i \sim P(\mu_i) and pull the arm with the highest sample. Empirically, Thompson Sampling often outperforms UCB in practice because it balances exploration more smoothly.

Counterfactual evaluation

How do you evaluate whether exploration helped, without running a live A/B test? Inverse Propensity Scoring (IPS) reweights offline logged data to estimate counterfactual policy performance:

R^(π)=1N∑t=1Nπ(at∣st)π0(at∣st) rt\hat{R}(\pi) = \frac{1}{N}\sum_{t=1}^N \frac{\pi(a_t | s_t)}{\pi_0(a_t | s_t)}\, r_t

where π0\pi_0 is the logging policy and π\pi is the new policy being evaluated. Items the new policy would have chosen but the logging policy rarely showed get upweighted. Netflix and Lyft use IPS-style offline evaluation to estimate the lift from exploration strategies before committing to an A/B test.

Exercise

You apply Maximal Marginal Relevance (MMR) with λ = 0 to a candidate list. What does the resulting ranked list optimize for?

Need a hint?

Think about what λ = 0 and λ = 1 mean in the MMR formula.

Exercise

In Thompson Sampling for a multi-armed bandit, what does 'sampling from the posterior' accomplish?

Need a hint?

Compare Thompson Sampling to a deterministic algorithm like UCB.

Exercise

A new item has zero interaction history on your platform. Which cold-start strategy provides the most immediate signal without requiring any user interaction data for that item?

Need a hint?

Think about what's available for a brand-new item.

Common mistakes

  1. Optimizing NDCG alone. Accuracy-only ranking converges on the dominant taste cluster — measure coverage and intra-list diversity alongside relevance.
  2. Onboarding with the most popular items. Popular items are the least discriminative; ask about items that partition the preference space.
  3. Evaluating a new policy only on logged data without propensities. Naive offline replay is biased toward the logging policy; use IPS (or an A/B test).

Key takeaways

  • Cold start requires falling back to content-based similarity (features, not interactions); meta-learning and explicit onboarding signals are the best tools for new users.
  • Diversity is not a soft goal — filter bubbles reduce long-term retention. MMR (greedy sequential selection) and DPPs (probabilistic quality-plus-diversity) are the main algorithmic tools.
  • Exploration via UCB or Thompson Sampling prevents the model from being trapped by its own past recommendations; counterfactual evaluation (IPS) lets you assess exploration strategies offline.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

Builds onRelatedGo deeper
Diversity, Cold Start & Explorati…Linear Algebra for MLExploration Strategies & …Session-Based & Real-Time…Q-LearningAd Ranking & CTR Predicti…The Cold Start Problem in…Ad Auction Mechanisms & S…