Diversity, Cold Start & Exploration
Optimizing NDCG@10 can produce a recommender that is technically accurate but practically terrible: it recommends the same ten items to everyone, ignores new users and products, and traps users in an ever-narrowing taste bubble. This lesson covers the three structural problems every production recommender must address beyond raw accuracy.
Real-world examples: Spotify's "Discover Weekly" (diversity by design), Netflix's causal inference for exploration, Pinterest's cold-start for new Pins, LinkedIn's cold-start for new job listings, Airbnb's neighborhood for new property listings.
The cold start problem
Cold start occurs when a user or item has too few interactions to produce reliable embeddings. It comes in three flavors:
| Flavor | Problem | Example |
|---|---|---|
| New user | No interaction history | A just-signed-up user on Spotify |
| New item | No engagement data yet | A freshly listed product on Etsy |
| New context | Unseen feature combination | A user in a new geographic market |
Content-based fallback
The most practical cold-start remedy: represent items by their features (text description, category, price, image embedding) rather than interaction-derived embeddings. When interaction data is sparse, fall back to content similarity.
For a new item with features , project to the embedding space:
The projection is learned during training to align content features with interaction-trained embeddings. At Airbnb, new listings' content embeddings (images, description text, amenities) are used immediately on day 0; they converge to interaction-based embeddings over ~2 weeks as bookings accumulate.
Meta-learning for fast user adaptation
MAML (Model-Agnostic Meta-Learning) trains a base model such that a small number of gradient steps on a new user's first few interactions produces a high-quality personalized model. In recommendation:
The inner loop updates for user on their first interactions; the outer loop optimizes the initialization to be maximally "fast-adaptable". Effective for platforms where users are highly heterogeneous and the onboarding sequence is short.
Onboarding intent capture
Many platforms explicitly ask new users for preferences at signup: Spotify's "pick your favorite artists," Tinder's "swipe on these profiles." These explicit signals bootstrap the embedding even before any implicit interaction occurs. The key is to show items that maximally partition the latent preference space (items that are highly discriminative), not just the most popular ones.
Warm-up strategy at Netflix
Netflix uses a combination: new users see a set of titles chosen to elicit a preference signal (explicit rating request), and in parallel are assigned to a "taste cluster" of existing users whose early ratings matched the new user's profile. The cluster's collaborative signal is used while the new user's personal data accumulates.
Diversity beyond accuracy
A ranker optimizing average relevance will recommend the best item over and over — serving only the dominant taste cluster and creating a filter bubble. Diversity metrics measure how varied a recommendation list is:
- Intra-list diversity (ILD): average pairwise dissimilarity of recommended items. .
- Coverage: fraction of the item catalog that ever appears in recommendations across users.
- Serendipity: relevance-weighted surprise — items that are both liked and unexpected.
Maximal Marginal Relevance (MMR)
MMR re-ranks a candidate list to balance relevance and novelty. At each step, select the item that maximizes:
where is the relevance score, is the set of already-selected items, and is embedding cosine similarity. trades off relevance vs diversity. A greedy selection of the top- items via MMR takes and is trivially parallelizable.
Spotify applies MMR-style diversification at the re-ranking stage: after retrieval and ranking, the list is passed through a diversity optimizer that ensures no two consecutive songs share the same artist, genre, or BPM bracket.
Determinantal Point Processes (DPP)
DPPs assign higher probability to subsets of items that are both high quality and mutually diverse:
where (the kernel matrix). The determinant is large when items have high quality ( large) and low mutual similarity (off-diagonal terms small). DPPs are mathematically elegant but expensive to sample from exactly; in practice, greedy approximations achieve > 90% of optimal diversity in .
Exploration vs exploitation
Even with a well-trained model, there are items the model thinks aren't relevant but would actually be enjoyed, because there's no data to learn from. Exploration is the deliberate injection of uncertainty-resolved queries.
Multi-armed bandits recap
The bandit problem (treated in full in Exploration Strategies & Model-Based RL): arms with unknown reward distributions. At each step, pull an arm to get a reward and update beliefs. The fundamental tension: exploit the current best arm, or explore an uncertain arm to reduce uncertainty.
Upper Confidence Bound (UCB): Add an exploration bonus proportional to uncertainty:
Items pulled fewer times ( small) get larger bonuses, ensuring every item gets explored eventually. UCB is provably sublinear in regret for stationary distributions.
Thompson Sampling: Maintain a posterior distribution over each arm's mean. At each step, sample and pull the arm with the highest sample. Empirically, Thompson Sampling often outperforms UCB in practice because it balances exploration more smoothly.
Counterfactual evaluation
How do you evaluate whether exploration helped, without running a live A/B test? Inverse Propensity Scoring (IPS) reweights offline logged data to estimate counterfactual policy performance:
where is the logging policy and is the new policy being evaluated. Items the new policy would have chosen but the logging policy rarely showed get upweighted. Netflix and Lyft use IPS-style offline evaluation to estimate the lift from exploration strategies before committing to an A/B test.
Exercise
You apply Maximal Marginal Relevance (MMR) with λ = 0 to a candidate list. What does the resulting ranked list optimize for?
Need a hint?
Think about what λ = 0 and λ = 1 mean in the MMR formula.
Exercise
In Thompson Sampling for a multi-armed bandit, what does 'sampling from the posterior' accomplish?
Need a hint?
Compare Thompson Sampling to a deterministic algorithm like UCB.
Exercise
A new item has zero interaction history on your platform. Which cold-start strategy provides the most immediate signal without requiring any user interaction data for that item?
Need a hint?
Think about what's available for a brand-new item.
Common mistakes
- Optimizing NDCG alone. Accuracy-only ranking converges on the dominant taste cluster — measure coverage and intra-list diversity alongside relevance.
- Onboarding with the most popular items. Popular items are the least discriminative; ask about items that partition the preference space.
- Evaluating a new policy only on logged data without propensities. Naive offline replay is biased toward the logging policy; use IPS (or an A/B test).
Key takeaways
- Cold start requires falling back to content-based similarity (features, not interactions); meta-learning and explicit onboarding signals are the best tools for new users.
- Diversity is not a soft goal — filter bubbles reduce long-term retention. MMR (greedy sequential selection) and DPPs (probabilistic quality-plus-diversity) are the main algorithmic tools.
- Exploration via UCB or Thompson Sampling prevents the model from being trapped by its own past recommendations; counterfactual evaluation (IPS) lets you assess exploration strategies offline.
Related concepts
- Session-Based & Real-Time Recommendations: the sequential model that cold-start content embeddings plug into
- Q-Learning: exploration/exploitation is the same problem as in RL
- Ad Ranking & CTR Prediction: ad systems add budget and auction constraints on top of relevance