KNN & Decision Trees Quiz

KNN & Decision Trees Quiz

Check your understanding of instance-based and tree-based learning — the bias–variance effect of k, why scaling matters for distances, and how trees choose their splits.

Exercise

What happens to a KNN classifier as k decreases to 1?

Need a hint?

With k = 1, the training error is exactly zero. Suspicious?

Exercise

Why does KNN usually require feature scaling?

Need a hint?

Imagine one feature in meters (0–2) and another in dollars (0–100,000).

Exercise

Information gain measures what about a candidate split?

Need a hint?

Entropy before vs. entropy after.

Exercise

What is the Gini impurity of a node containing only one class?

Need a hint?

Gini = 1 − Σpᵢ². What if one pᵢ = 1?

Exercise

An unconstrained decision tree grown to purity usually does what?

Need a hint?

Each leaf may end up holding a single training point.

Exercise

Increasing k in KNN (or capping a decision tree's depth) tunes model complexity. Which slot of the ML project loop does this dial live in?

Need a hint?

You are changing how expressive the model family is, not the loss or the data.

Exercise

Transfer test — a technique this course did NOT teach: focal loss down-weights easy examples so a detector focuses on hard, rare ones. Which slot does it modify?

Need a hint?

It reshapes the loss to care about different errors.