Building with LLMs Quiz

Building with LLMs Quiz

Check your understanding of the full stack — how a frozen model adapts from the prompt, what decoding knobs do, how semantic search and RAG ground answers in your data, and how to keep a production LLM app safe.

Exercise

Few-shot prompting improves a frozen model's task performance by which mechanism?

Need a hint?

Are any weights updated?

Exercise

Lowering temperature toward 0 has what effect on decoding?

Need a hint?

What happens to the softmax over logits as T → 0?

Exercise

Which similarity measure underlies most semantic search over text embeddings?

Need a hint?

It ignores vector magnitude and looks at direction.

Exercise

In a RAG pipeline, what is the correct order of the query-time steps?

Need a hint?

The model only sees the chunks after they're fetched.

Exercise

Which is the most effective defense against indirect prompt injection in an LLM application?

Need a hint?

The attack works because data is treated as instructions.

Exercise

An agent's individual steps each succeed 95% of the time. Roughly how reliable is a 10-step task, and what follows for agent design?

Need a hint?

0.95¹⁰ ≈ ?

Exercise

What distinguishes a reasoning model (o1, DeepSeek-R1) from a base LLM given chain-of-thought prompting?

Need a hint?

One completes a pattern; the other was trained with RL on verified answers.

Exercise

You use GPT-class model as a judge to compare answers A and B. Which safeguard addresses a known, measured bias of LLM judges?

Need a hint?

What happens if the judge always sees the candidate answers in the same order?

Exercise

Why do code models (Copilot, StarCoder2) train with fill-in-the-middle (FIM) rather than only left-to-right completion?

Need a hint?

Where is a developer's cursor, usually?

Exercise

Adding retrieval-augmented generation (RAG) — fetching documents and inserting them into the prompt — changes which slot of the agentic project loop?

Need a hint?

It changes what information the model has access to at inference time.

Exercise

Transfer test — a technique this course did NOT teach: an agent that critiques and revises its own draft before answering. Which slot of the agentic project loop does self-reflection modify?

Need a hint?

It adds a step to the runtime loop the agent runs.