Session-Based Recommendation Algorithms
Session-based recommendation models treat a user's current browsing sequence as a first-class signal, predicting the next item from the ordered list of items interacted with in the current session. This page walks through the three dominant architectures: GRU4Rec, SASRec, and BERT4Rec.
Problem formulation
Given a session (an ordered list of item IDs), predict the next item from an item catalog of size .
Training objective: maximize the probability of the observed next item:
GRU4Rec
Architecture: Item embedding → GRU → item score projection.
Training: BPR-max loss or cross-entropy with in-session negatives. For a positive item and negative samples :
Strengths: Fast, low memory, works well for short sessions (< 20 items).
Limitations: Sequential bottleneck (hidden state compresses all history), poor on long sessions.
SASRec (Self-Attentive Sequential Recommendation)
Architecture: Item embedding → positional encoding → causal transformer layers → next-item head.
Causal self-attention in SASRec
Position embeddings are added to item embeddings:
Each transformer layer applies multi-head causal (masked) self-attention:
where if (future masking), otherwise. This preserves the left-to-right ordering while allowing attention to attend to any previous position.
After layers, the representation at position is used to score all items:
Training with binary cross-entropy
SASRec uses a binary cross-entropy loss per position, with one randomly sampled negative item per positive:
Hyperparameter guide
| Hyperparameter | Typical value | Impact |
|---|---|---|
| Number of layers | 2 | Depth vs. overfitting |
| Embedding dim | 64 | Quality vs. memory |
| Max sequence length | 50–200 | Truncate from right (most recent) |
| Dropout | 0.2 | Regularization for sparse datasets |
BERT4Rec
Key difference from SASRec: bidirectional attention via Masked Item Modeling (MIM).
Training: Randomly mask 15–20% of items in the sequence; predict the masked items from both left and right context:
After a BERT-style encoder (bidirectional self-attention), predict the masked item:
Serving: At inference time, append a [MASK] token to the end of the session and predict the next item from the masked position.
When to prefer BERT4Rec over SASRec:
- Tasks where you need to infer items from context in both directions (e.g., fill-in-the-list).
- Datasets where future context is available at inference time (e.g., playlist continuation, not real-time next-click prediction).
Comparison table
| GRU4Rec | SASRec | BERT4Rec | |
|---|---|---|---|
| Architecture | GRU | Causal Transformer | BERT Transformer |
| Attention | None | Left-to-right | Bidirectional |
| Training loss | BPR-max | Binary CE | MLM |
| Best for | Short sessions, fast training | General next-item | Fill-in / context-rich tasks |
| Inference | Sequential | attention | attention + one mask |
Negative sampling strategies
| Strategy | Description | Quality |
|---|---|---|
| Random | Sample from full catalog uniformly | Fast, weak signal |
| In-batch | Other items in the same training batch | Efficient, good for large catalogs |
| Popularity-based | Sample proportional to item frequency | Harder negatives, more realistic |
| Hard negative mining | Sample items semantically close to the positive | Best signal, expensive |
For SASRec-style models, in-batch negatives (treating other sequence items in the batch as negatives) are the standard trade-off between compute and signal quality.
Related concepts
- Session-Based & Real-Time Recommendations: the lesson this walks through in algorithm detail
- Self-Attention: the causal attention SASRec and BERT4Rec are built on
- The Cold Start Problem: session models as a cold-start remedy for new users