Ensemble Methods Quiz

Ensemble Methods Quiz

Check your understanding of ensembles, which error component each method attacks, what makes Random Forests more than bagged trees, and what gradient boosting fits at each round.

Exercise

Bagging primarily reduces which component of error?

Need a hint?

Averaging many noisy estimates does what to the noise?

Exercise

How does boosting differ fundamentally from bagging?

Need a hint?

Can you train boosted trees in parallel?

Exercise

Beyond bagging, what extra randomness does a Random Forest add?

Need a hint?

Look at what each split is allowed to consider.

Exercise

What are out-of-bag (OOB) samples used for?

Need a hint?

Each bootstrap sample leaves out ~37% of the training points.

Exercise

In gradient boosting, what does each new tree fit?

Need a hint?

For squared error, this is just the residuals.

Exercise

Gradient boosting adds each tree by taking a gradient step in function space. Which slot of the ML project loop is that fundamentally an instance of?

Need a hint?

It's a search that drives the ensemble downhill on the loss, one step per round.

Exercise

Transfer test — a technique this course did NOT teach: dropout randomly zeroes activations during training so a big network can't memorize. Which slot does it modify?

Need a hint?

It restricts the effective capacity of the model family.