NLP Quiz
Check your understanding across the course — BPE and embeddings, BERT pre-training, picking a model family, when beam search fails, why BERTopic reduces before clustering, and the bi-encoder/cross-encoder split.
Exercise
Which property of BPE (Byte-Pair Encoding) directly solves the out-of-vocabulary (OOV) problem that word-level tokenization suffers from?
Need a hint?
BPE starts from characters — what can it always fall back to?
Exercise
GloVe (Global Vectors) trains on a different kind of statistics than Word2Vec. What is GloVe's training signal?
Need a hint?
The 'Global' in GloVe is a clue — it uses the full corpus, not local windows.
Exercise
In a seq2seq model, Bahdanau attention computes a context vector c_t = Σ α_{t,s} h_s at each decoding step. What problem does this directly solve compared to the original seq2seq model?
Need a hint?
The original seq2seq uses a single fixed c for all decoding steps.
Exercise
BERT is pre-trained with Next Sentence Prediction (NSP): given two segments A and B, predict whether B follows A in the original text. What was NSP intended to teach BERT, and what did RoBERTa later find?
Need a hint?
NSP's purpose was inter-sentence understanding. RoBERTa's finding was...
Exercise
When fine-tuning BERT for a new task, what is the typical fine-tuning strategy for the model's weights?
Need a hint?
BERT fine-tuning tunes all weights, not just the task head — this is 'full fine-tuning'.
Exercise
You need to embed 10M support tickets for semantic search, and separately need a chatbot that drafts replies. Which model families fit?
Need a hint?
Representations vs generation — which mask does each need?
Exercise
Why does beam search — which finds higher-probability sequences than sampling — produce worse open-ended chat and stories?
Need a hint?
Is the most likely string of English also typical English?
Exercise
In the BERTopic pipeline, why is UMAP dimensionality reduction run before HDBSCAN clustering rather than clustering the raw 768-dim embeddings?
Need a hint?
What happens to pairwise distances as dimension grows?
Exercise
Why do retrieval systems use a bi-encoder for first-stage search and save the more accurate cross-encoder for reranking only?
Need a hint?
Which architecture lets you precompute document vectors offline?
Exercise
Tokenization and embeddings turn text into numbers before any language model sees it. Which slot of the ML project loop do these choices live in?
Need a hint?
They define the representation the model consumes — upstream of the architecture.
Exercise
Transfer test — a technique this course did NOT teach: an input guardrail that blocks prompt-injection attempts before they reach an LLM. In the agentic project loop, which slot does it belong to?
Need a hint?
It contains a failure mode at runtime by filtering what enters the system.