The Recommendation Problem
Every feed, store, and streaming service answers the same question millions of times a second: of all the items, which few should I show this user now? Recommender systems are the ML behind that choice, and they're among the highest-value ML applications in industry. This lesson frames the problem, contrasts the two classic families of solution, and defines how we actually measure a recommendation. Recommenders stress the far end of the ML project loop: getting the objective and ranking evaluation right (recall@k, NDCG) matters as much as the model, and because recommendations shape the very clicks you train on next, the deployment-feedback loop is unusually tight.
Real-world examples: Netflix's "Top picks," Amazon's "Customers also bought," Spotify's Discover Weekly, YouTube's up-next, and the ranking of any social feed.
The setup
You have a set of users and items , and a sparse record of past interactions. The goal is to predict, for a user, the items they'd most likely engage with, and surface a short ranked list.
The interaction data is a user–item matrix , where is user 's response to item . Crucially, this matrix is extremely sparse: a user has touched a tiny fraction of all items, so most entries are unknown (not zero). Recommendation is, at heart, filling in those unknowns.
Explicit vs. implicit feedback
| Explicit | Implicit | |
|---|---|---|
| Signal | ratings, thumbs, reviews | clicks, watches, purchases, dwell time |
| Meaning | stated preference | inferred preference |
| Volume | sparse, costly | abundant, cheap |
| Negatives | a low rating is real dislike | a non-click is ambiguous (unseen? disliked?) |
Implicit feedback has no true negatives
A user not clicking an item usually means they never saw it — not that they dislike it. So you can't treat un-interacted items as confirmed negatives. Implicit-feedback models use techniques like negative sampling and confidence weighting (interactions are positive signals of varying strength) rather than assuming missing = negative.
Two classic families
Content-based filtering recommends items similar to ones the user liked, using item features (genre, text, tags). Build a profile vector from a user's history, then rank items by feature similarity (e.g. cosine).
- ✅ Works for brand-new items (uses their features) and needs no other users.
- ❌ Stuck in a "filter bubble" (only ever recommends more of the same); needs good item features.
Collaborative filtering (CF) recommends what similar users liked, using only the interaction matrix — no item features at all. "Users who liked what you liked also liked X."
- ✅ Discovers surprising cross-domain recommendations; no feature engineering.
- ❌ Suffers cold start: a brand-new user or item has no interactions to learn from.
Memory-based vs. model-based CF
Early CF was memory-based: find the k nearest users (or items) by interaction overlap and average their preferences — essentially KNN on the rating matrix. Modern CF is model-based: learn low-dimensional latent factors that generate the matrix (matrix factorization, next lesson). Production systems usually hybridize content + collaborative signals to get the best of both and soften cold start.
The cold-start problem
Collaborative filtering can't recommend for entities with no interaction history:
- New user: no history to find similar users. Mitigate with onboarding questions, popularity defaults, or content/context features.
- New item: no one has interacted with it. Mitigate with content-based features until interactions accumulate.
Cold start is the reason pure CF is rarely deployed alone.
Measuring a recommender: ranking metrics
A recommender outputs a ranked list, so accuracy-style metrics miss the point — what matters is whether relevant items appear near the top. The standard top- metrics:
- Precision@k: fraction of the top- recommendations that are relevant.
- Recall@k: fraction of all relevant items that appear in the top-.
- NDCG@k (Normalized Discounted Cumulative Gain) — rewards putting relevant items higher, with a logarithmic position discount:
where is the DCG of the ideal (perfectly sorted) ranking, so NDCG . MRR (mean reciprocal rank) and MAP are also common.
Worked example. Top-3 list where items 1 and 3 are relevant () and item 2 is not: . The ideal ordering puts both relevant items first: . So — the misplaced second slot costs 8%.
Why position-aware metrics?
Two recommenders can have identical recall@10 yet very different user experience if one puts the relevant item at rank 1 and the other at rank 10. NDCG and MRR capture that the order within the list matters — the top slot is worth far more than the tenth.
Five metrics, one list
Press the first two presets in turn. P@10 does not move — it counts a set, and the set is identical — while NDCG falls from 1.0000 to 0.3452 and MRR from 1.0000 to 0.1250. Same documents, same recall, a ranking nobody would ship.
Click a row to cycle its relevance grade (0–3). Every metric is recomputed from the grades: DCG = Σ (2^rel − 1)/log₂(i+2) with NDCG normalising by the ideal ordering, MAP averaging precision at each hit, MRR reading only the first. Bar length is the item's gain; position is its rank.
That callout is a claim you can make numeric in two clicks. Press relevant at the top, then same items, at the bottom — the list contains exactly the same three documents, so precision and recall cannot tell them apart:
| P@10 | P@3 | MRR | MAP | NDCG@10 | |
|---|---|---|---|---|---|
| relevant at the top | 0.300 | 1.000 | 1.0000 | 1.0000 | 1.0000 |
| same items, at the bottom | 0.300 | 0.000 | 0.1250 | 0.2157 | 0.3452 |
does not move at all. NDCG falls and MRR . A metric that counts a set is blind to the only thing a ranker controls, which is why "we improved recall@100" is not by itself evidence that anyone's feed got better.
Reading the gains column
Each row's contribution is : the numerator makes a grade-3 item worth 7 and a grade-1 item worth 1, and the denominator discounts by position. Moving a perfect item from rank 1 to rank 8 keeps its relevance and divides its contribution by 3.17.
Exercise
You train a recommender on click logs (implicit feedback). Why is it wrong to treat every item a user did NOT click as a negative example?
Need a hint?
Did the user even see those items?
Exercise
Pure collaborative filtering cannot make good recommendations for a brand-new item with no interactions. What is this problem called, and a standard fix?
Need a hint?
It also affects brand-new users.
Exercise
Two recommenders both place exactly the same 3 relevant items somewhere in their top-10, so they have identical recall@10. Why might their NDCG@10 differ?
Need a hint?
What does the log discount in DCG reward?
Common mistakes
- Treating missing implicit feedback as a negative. A non-click is ambiguous; use negative sampling, not "missing = dislike."
- Evaluating with RMSE only. Rating-prediction error ignores ranking quality; report top- ranking metrics (recall@k, NDCG).
- Ignoring cold start. A pure-CF demo looks great offline but fails the moment a new user or item arrives.
- Leaking the future. Splitting interactions randomly lets the model "see" later behavior; split by time to mimic deployment.
Key takeaways
- Recommendation = filling in a sparse user–item matrix and surfacing a short ranked list.
- Explicit feedback (ratings) is sparse but clear; implicit feedback (clicks) is abundant but has no true negatives.
- Content-based filtering uses item features (handles new items, risks filter bubbles); collaborative filtering uses the interaction matrix (discovers novel items, suffers cold start). Production systems hybridize.
- Judge recommenders with position-aware ranking metrics (recall@k, NDCG@k, MRR), not plain accuracy or RMSE.
Related concepts
- Matrix Factorization: the model-based CF workhorse that learns latent user/item factors
- KNN: the nearest-neighbor idea behind memory-based collaborative filtering
- Classification Metrics: precision/recall, which the top-k ranking metrics extend
- The Cold Start Problem: strategies for new users, items, and contexts with no interaction history