Session-Based Recommendation AlgorithmsOpen in Colab

Session-Based Recommendation Algorithms

Session-based recommendation models treat a user's current browsing sequence as a first-class signal, predicting the next item from the ordered list of items interacted with in the current session. This page walks through the three dominant architectures: GRU4Rec, SASRec, and BERT4Rec.

Problem formulation

Given a session S=[i1,i2,…,iT]S = [i_1, i_2, \ldots, i_T] (an ordered list of item IDs), predict the next item iT+1i_{T+1} from an item catalog of size ∣I∣|I|.

Training objective: maximize the probability of the observed next item:

L=−∑t=1T−1log⁡P(it+1∣i1,…,it;θ)\mathcal{L} = -\sum_{t=1}^{T-1} \log P(i_{t+1} \mid i_1, \ldots, i_t; \theta)

GRU4Rec

Architecture: Item embedding → GRU → item score projection.

et=Embed(it),ht=GRU(et,ht−1),y^t=Wht\mathbf{e}_t = \text{Embed}(i_t), \quad \mathbf{h}_t = \text{GRU}(\mathbf{e}_t, \mathbf{h}_{t-1}), \quad \hat{\mathbf{y}}_t = W \mathbf{h}_t

Training: BPR-max loss or cross-entropy with in-session negatives. For a positive item i+i^+ and kk negative samples {ij−}\{i^-_j\}:

LBPR=−log⁡σ ⁣(y^i+−1k∑jy^ij−)\mathcal{L}_{\text{BPR}} = -\log \sigma\!\left(\hat{y}_{i^+} - \frac{1}{k}\sum_j \hat{y}_{i^-_j}\right)

Strengths: Fast, low memory, works well for short sessions (< 20 items).
Limitations: Sequential bottleneck (hidden state compresses all history), poor on long sessions.

SASRec (Self-Attentive Sequential Recommendation)

Architecture: Item embedding → positional encoding → LL causal transformer layers → next-item head.

Causal self-attention in SASRec

Position embeddings are added to item embeddings:

E(0)=[e1+p1,  …,  eT+pT]\mathbf{E}^{(0)} = [\mathbf{e}_1 + \mathbf{p}_1,\; \ldots,\; \mathbf{e}_T + \mathbf{p}_T]

Each transformer layer applies multi-head causal (masked) self-attention:

Attention(Q,K,V)=softmax ⁣(QK⊤d+M)V\text{Attention}(Q, K, V) = \text{softmax}\!\left(\frac{QK^\top}{\sqrt{d}} + M\right) V

where Mij=−∞M_{ij} = -\infty if j>ij > i (future masking), 00 otherwise. This preserves the left-to-right ordering while allowing attention to attend to any previous position.

After LL layers, the representation at position TT is used to score all items:

score(iT+1)=hT(L)⋅eiT+1\text{score}(i_{T+1}) = \mathbf{h}_T^{(L)} \cdot \mathbf{e}_{i_{T+1}}

Training with binary cross-entropy

SASRec uses a binary cross-entropy loss per position, with one randomly sampled negative item per positive:

L=−∑t=1T−1[log⁡σ(y^it+1)+log⁡(1−σ(y^it+1−))]\mathcal{L} = -\sum_{t=1}^{T-1}\left[\log \sigma(\hat{y}_{i_{t+1}}) + \log(1 - \sigma(\hat{y}_{i_{t+1}^-}))\right]

Hyperparameter guide

HyperparameterTypical valueImpact
Number of layers LL2Depth vs. overfitting
Embedding dim dd64Quality vs. memory
Max sequence length50–200Truncate from right (most recent)
Dropout0.2Regularization for sparse datasets

BERT4Rec

Key difference from SASRec: bidirectional attention via Masked Item Modeling (MIM).

Training: Randomly mask 15–20% of items in the sequence; predict the masked items from both left and right context:

E(0)=[e1+p1,  …,  [MASK]+pt,  …]\mathbf{E}^{(0)} = [\mathbf{e}_1 + \mathbf{p}_1,\; \ldots,\; [\text{MASK}] + \mathbf{p}_t,\; \ldots]

After a BERT-style encoder (bidirectional self-attention), predict the masked item:

i^t=arg⁡max⁡softmax ⁣(Woutht(L))\hat{i}_t = \arg\max \text{softmax}\!\left(W_{\text{out}} \mathbf{h}_t^{(L)}\right)

Serving: At inference time, append a [MASK] token to the end of the session and predict the next item from the masked position.

When to prefer BERT4Rec over SASRec:

  • Tasks where you need to infer items from context in both directions (e.g., fill-in-the-list).
  • Datasets where future context is available at inference time (e.g., playlist continuation, not real-time next-click prediction).

Comparison table

GRU4RecSASRecBERT4Rec
ArchitectureGRUCausal TransformerBERT Transformer
AttentionNoneLeft-to-rightBidirectional
Training lossBPR-maxBinary CEMLM
Best forShort sessions, fast trainingGeneral next-itemFill-in / context-rich tasks
InferenceSequentialO(T2)O(T^2) attentionO(T2)O(T^2) attention + one mask

Negative sampling strategies

StrategyDescriptionQuality
RandomSample from full catalog uniformlyFast, weak signal
In-batchOther items in the same training batchEfficient, good for large catalogs
Popularity-basedSample proportional to item frequencyHarder negatives, more realistic
Hard negative miningSample items semantically close to the positiveBest signal, expensive

For SASRec-style models, in-batch negatives (treating other sequence items in the batch as negatives) are the standard trade-off between compute and signal quality.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
Session-Based Recommendation Algo…The Cold Start Problem in…Session-Based & Real-Time…Self-Attention