NLP Quiz

NLP Quiz

Check your understanding across the course — BPE and embeddings, BERT pre-training, picking a model family, when beam search fails, why BERTopic reduces before clustering, and the bi-encoder/cross-encoder split.

Exercise

Which property of BPE (Byte-Pair Encoding) directly solves the out-of-vocabulary (OOV) problem that word-level tokenization suffers from?

Need a hint?

BPE starts from characters — what can it always fall back to?

Exercise

GloVe (Global Vectors) trains on a different kind of statistics than Word2Vec. What is GloVe's training signal?

Need a hint?

The 'Global' in GloVe is a clue — it uses the full corpus, not local windows.

Exercise

In a seq2seq model, Bahdanau attention computes a context vector c_t = Σ α_{t,s} h_s at each decoding step. What problem does this directly solve compared to the original seq2seq model?

Need a hint?

The original seq2seq uses a single fixed c for all decoding steps.

Exercise

BERT is pre-trained with Next Sentence Prediction (NSP): given two segments A and B, predict whether B follows A in the original text. What was NSP intended to teach BERT, and what did RoBERTa later find?

Need a hint?

NSP's purpose was inter-sentence understanding. RoBERTa's finding was...

Exercise

When fine-tuning BERT for a new task, what is the typical fine-tuning strategy for the model's weights?

Need a hint?

BERT fine-tuning tunes all weights, not just the task head — this is 'full fine-tuning'.

Exercise

You need to embed 10M support tickets for semantic search, and separately need a chatbot that drafts replies. Which model families fit?

Need a hint?

Representations vs generation — which mask does each need?

Exercise

Why does beam search — which finds higher-probability sequences than sampling — produce worse open-ended chat and stories?

Need a hint?

Is the most likely string of English also typical English?

Exercise

In the BERTopic pipeline, why is UMAP dimensionality reduction run before HDBSCAN clustering rather than clustering the raw 768-dim embeddings?

Need a hint?

What happens to pairwise distances as dimension grows?

Exercise

Why do retrieval systems use a bi-encoder for first-stage search and save the more accurate cross-encoder for reranking only?

Need a hint?

Which architecture lets you precompute document vectors offline?

Exercise

Tokenization and embeddings turn text into numbers before any language model sees it. Which slot of the ML project loop do these choices live in?

Need a hint?

They define the representation the model consumes — upstream of the architecture.

Exercise

Transfer test — a technique this course did NOT teach: an input guardrail that blocks prompt-injection attempts before they reach an LLM. In the agentic project loop, which slot does it belong to?

Need a hint?

It contains a failure mode at runtime by filtering what enters the system.