Design a Home-Feed Ranking System
ML project loop
Data·Hypothesis space·Objective·Optimization·Evaluation·Deployment feedback

Design a Home-Feed Ranking System

Track

ML System Design

Inspired by

Pinterest / Netflix

Scale

300M+ users · billions of items · <200ms budget

Difficulty

Intermediate

"Design the home feed" is the canonical ML system-design interview. The trap is to reach for a model on the first sentence. Strong answers move through the ML project loop in order — data → hypothesis space → objective → optimization → evaluation → feedback, and treat every design choice as which slot am I filling, and what breaks if I skip it? The SpineNav strip above marks the stages this problem leans on hardest. The design below is not hypothetical — it is, to a first approximation, how Instagram Explore, YouTube's recommender, and Twitter's For You feed are actually built, and each stage is anchored in what those teams have published.

1. Clarify the problem

Before designing anything, pin down scope with the interviewer:

  • What is the feed? A ranked, infinite-scroll list of items (pins, videos, posts) personalized per user, refreshed on each session.
  • What are we optimizing? Long-term engagement (return visits, time well spent) — not raw click-through, which is easy to game with clickbait. YouTube made this concrete by ranking on expected watch time rather than click probability, precisely because clickbait wins clicks it can't hold (Covington et al., RecSys 2016).
  • Who are the users / items? 300M+ users, billions of candidate items, heavy-tailed popularity, constant cold-start on both sides.
  • What's out of scope? Ads ranking (a separate auction — see the ad-ranking lesson), notifications, and search.

Interview signal

Naming the objective as long-term engagement and immediately flagging the proxy-metric risk (CTR → clickbait) is the single highest-signal thing you can say in the first two minutes.

Think firstBillions of candidate items, a 200ms end-to-end budget. Before any modeling — what does the arithmetic alone force on the architecture?Commit to an answer, then reveal

Divide the budget by the scale: at 10k QPS you can afford to run an expensive model over a few hundred candidates per request, not billions. Something cheap has to shrink billions down to hundreds first — the arithmetic forces a funnel before you've picked a single model.

2. Requirements & scale

DimensionTargetConsequence for the design
Latency< 200 ms p99 end-to-endRules out scoring billions of items directly: forces a funnel
Throughput~10k feed requests/sec at peakRanker must be cheap per item; heavy work goes offline
FreshnessNew items rankable within minutesStreaming feature pipeline, not nightly batch
Candidate poolBillions of itemsApproximate nearest-neighbor retrieval, not brute force
Availability99.9%Graceful fallback to a non-personalized feed

Back-of-envelope: 10k QPS × 200ms budget means each request scores maybe a few hundred candidates through a heavy model, not millions. That single constraint dictates the whole architecture: a funnel. For calibration on what the top end looks like: Meta reports that Instagram's recommendation systems extract 65 billion features and make 90 million model predictions every second (Meta Engineering).

3. ML problem framing

The feed is not one model — it's a multi-stage retrieval → ranking pipeline, each stage a different ML slot:

  1. Retrieval (hypothesis space): from billions of items, cheaply fetch ~1000 candidates likely to be relevant. Framed as approximate nearest neighbor search in an embedding space.
  2. Ranking (hypothesis space + objective): score those ~1000 candidates with an expensive model predicting P(engage), framed as learning to rank / pointwise CTR-style prediction with an engagement objective.
  3. Re-ranking: apply business logic: diversity, freshness, dedup, policy.

Production systems make the funnel even deeper. Instagram Explore retrieves thousands of candidates from many sources, cuts them to hundreds with a lightweight first-stage ranker, and only then pays for the heavy model; YouTube's classic design narrows millions of videos to hundreds in candidate generation and ranks only dozens at the final stage. Each extra stage buys the next one a smaller, better pool.

What was breaking before? A single ranker over all items blows the latency budget; pure popularity ignores personalization. The funnel exists to make personalized scoring tractable under the budget.

Think firstNobody rates their feed. Where do the training labels come from — and what's the catch?Commit to an answer, then reveal

Implicit feedback: impressions, clicks, saves, dwell, hides. Abundant but biased — users can only engage with what the old model chose to show them, so naive training just relearns yesterday's policy. You have to correct for the logging policy (propensity weighting, a randomized-exposure slice) or the loop feeds on itself.

4. Data & features (spine: data)

  • Labels: implicit feedback — impressions, clicks, saves, long dwell, hides. Weight positives by engagement depth; treat impressions-without-action as negatives (with care around position bias).
  • Features: user (long/short-term interest embeddings), item (content embedding, age, popularity), and context (time, device, session so far).
  • Feature store: the same pipeline computes features for training and serving to avoid training-serving skew — the classic production bug where offline metrics look great and online performance tanks.
  • Freshness: velocity/recency features stream in near-real-time so a minutes-old viral item can surface. A subtle trick from YouTube: include the age of the training example as a feature, so the model can separate "this item is good" from "this item was new when logged" — without it, the model implicitly ranks by historical popularity and demotes everything fresh.

Position & popularity bias

Logged feedback is biased by what the old model showed. Correct for it — inverse-propensity weighting, or reserving a small randomized-exposure slice — or the model just relearns its own past behavior.

Think firstRetrieval has ~40ms to find 1000 relevant items among billions. What shape must the model take for that to even be possible?Commit to an answer, then reveal

User and item can't interact inside the model at query time — that would mean scoring billions of pairs per request. Encode them separately, so item embeddings are precomputed and indexed offline, and query-time work collapses to one user embedding plus an approximate nearest-neighbor lookup. That constraint is exactly why the two-tower architecture exists, and why cross-attention retrieval is off the table.

5. Model (spine: hypothesis space + objective + optimization)

Retrieval: the two-tower model. A user tower and an item tower each map to the same dd-dimensional space; relevance is a dot product:

s(u,i)=eu⊤eis(u, i) = \mathbf{e}_u^\top \mathbf{e}_i

Item embeddings are precomputed offline and indexed in an ANN index; at request time we embed the user once and query the index. Start from a matrix factorization baseline to build intuition, then replace the towers with deep encoders. Train with in-batch negatives — Pinterest's home-feed two-tower uses batch sizes around 6000 so every positive gets thousands of negatives for free (Pinterest Engineering).

Matrix factorization: low-rank reconstruction

R (true ratings)≈U·Vᵀreconstruction (rank 1)
true rank3
RMSE0.474
params stored12 vs 36

The 6×6 rating matrix R (left) is genuinely rank-3. A model with rank k approximates it as U·Vᵀ — k latent factors per user and per item. Raise k: the reconstruction (right) approaches R and the RMSE falls to ~0 at k=3. A few latent 'tastes' reconstruct the whole matrix with far fewer numbers than storing it directly.

Ranking: a deep multi-task model. The ~1000 retrieved candidates go through a richer model (DLRM-style: sparse ID embeddings + feature crosses + MLP, where the embedding tables dominate the parameter count — Naumov et al., 2019). Production rankers are multi-task: one shared network with heads for P(click), P(save), P(share), P(hide), combined into a single score by a hand-tuned value formula. Twitter's open-sourced For You ranker made its weights public — a reply that draws the author back into conversation is worth on the order of 100× a like, and a report is a large negative (twitter/the-algorithm), so the value formula, not the model, is where product strategy lives. Baseline: logistic regression on hand-crafted features → prove the pipeline end-to-end before adding depth.

For the retrieval index, the ANN structure itself is worth knowing cold:

Deep dive
HNSW — the graph index behind fast retrieval
Includes runnable notebook
Think firstPeak traffic, and the heavy ranker breaches its 80ms budget on 5% of requests. What do those users see — an error, a spinner, or something else?Commit to an answer, then reveal

Neither. The funnel doubles as a degradation ladder: serve the first-stage (lightweight) scores without the heavy re-rank, fall back to a popularity-plus-recency feed, or replay the user's last cached ranking. Each rung trades personalization quality for availability, which is why 99.9% availability was a requirements-table row, not an afterthought. An empty feed costs you the session; a slightly worse feed costs you almost nothing.

6. Serving & scaling

Latency budget breakdown (of the 200ms):

StageBudgetHow
Feature fetch~30 msFeature store, cached user features
Retrieval (ANN)~40 msHNSW over precomputed item embeddings; multiple sources merged
Ranking~80 msBatch-score ~1000 candidates on GPU/optimized CPU
Re-rank + assemble~20 msDiversity, dedup, policy filters

Cache aggressively (user embeddings, item metadata); precompute item embeddings in a nightly + streaming hybrid; degrade gracefully to a popularity feed if the ranker times out.

Deep dive
Vector databases — serving the ANN index
Includes runnable notebook
Think firstYour new ranker lifts offline NDCG by 3%. Do you ship it?Commit to an answer, then reveal

Not yet. Offline gains are computed on logged data the old model produced — position bias, training-serving skew, and proxy-metric mismatch can each fake a win. Ship behind an A/B test on the real objective (long-term engagement) with guardrail metrics, and let that (not NDCG) be the judge.

7. Evaluation (spine: evaluation)

  • Offline: ranking metrics — NDCG@k, Recall@k for retrieval, AUC/logloss for the ranker. Offline gains are necessary but not sufficient.
  • Online: the real judge is an A/B test on engagement, with guardrail metrics (dwell quality, hide rate, creator diversity, latency) so you don't ship a CTR win that quietly harms the ecosystem. Netflix runs its recommender A/Bs against member retention and medium-term engagement and credits personalization with over $1B/year of retained value (Gomez-Uribe & Hunt, 2016) — the online judge is a business metric, not a ranking metric.
  • Counterfactual: off-policy estimators (IPS) to screen candidates before spending A/B traffic.
Think firstThought experiment: you freeze everything — no retrains, no embedding refreshes — for 90 days. Offline metrics on the old test set don't move. What actually decays in production, and in what order?Commit to an answer, then reveal

Retrieval rots first: items uploaded after the freeze have no learned embeddings, so the fresh half of the corpus is invisible within days. Then the ranker's feature distributions drift (seasonality, trends, new creators) and its calibration goes with them. The old test set can't detect any of this because it's drawn from the old world. The feed degrades into a greatest-hits archive of the quarter you froze, which is why retraining cadence is part of the architecture, not an ops detail.

8. Monitoring & feedback (spine: feedback)

  • Drift: monitor feature distributions and prediction distributions; user interests and item inventory shift constantly.
  • Retraining: ranker retrained daily (or continuously); embeddings refreshed as new items arrive. The loop closes back into data.
  • Cold start: new items get content-based embeddings + an exploration boost; new users get a popularity/onboarding feed until signal accrues.
Deep dive
Cold start — bootstrapping new users & items
Includes runnable notebook

9. Tradeoffs & alternatives

ChoiceAlternativeWhy the funnel wins here
Two-stage funnelOne big ranker over all itemsLatency budget makes full scoring impossible
Two-tower retrievalCross-attention retrievalTowers allow precomputed item embeddings + ANN; cross-attention can't scale to billions
Implicit feedbackExplicit ratingsImplicit is abundant; explicit is sparse and biased
Daily retrainFully online learningSimpler, more stable; online learning adds infra + failure modes

10. Interviewer follow-ups

How do you stop the feed from becoming a filter bubble?

Add an explicit diversity objective in re-ranking (e.g. determinantal point processes or a simple category/creator quota), and reserve exploration slots. Track creator- and topic-diversity as guardrail metrics in every A/B test so a pure-engagement win that collapses diversity is caught before launch.

A viral item is 10 minutes old — how does it get ranked?

Streaming features (velocity, recency) feed the ranker within minutes, and the item gets a content-based embedding at ingest so retrieval can surface it before it has any engagement history. An exploration boost gives it initial impressions to gather signal.

Offline NDCG went up but the A/B test is flat. What happened?

Usual suspects: training-serving skew (features differ online), position/ popularity bias the offline metric rewards but users don't, objective mismatch (you optimized CTR, users care about dwell), or the offline eval leaked future information. Check feature parity first.

Key takeaways

  • The latency budget is the architecture: it forces a retrieval → ranking funnel, not a single model.
  • Two-tower retrieval makes personalized scoring tractable by precomputing item embeddings and querying an ANN index.
  • Optimize long-term engagement, guard the proxy metrics, and let the A/B test — not offline NDCG — be the final judge.
  • The whole thing is a loop: production feedback becomes tomorrow's training data, so drift monitoring and retraining are part of the design, not an afterthought.

References