Neural Networks Quiz
Check your understanding of the key ideas from this course — why networks need non-linearity, what backprop computes, and how layers transform data. Use the hints if you get stuck.
Exercise
Why does a neural network need non-linear activation functions?
Need a hint?
What happens when you compose two linear functions?
Exercise
A dense layer with weights W, bias b, and activation σ computes which function of its input x?
Need a hint?
Linear transformation first, then the non-linearity.
Exercise
Why can't a single perceptron solve XOR?
Need a hint?
Try drawing the four XOR points and separating them with one line.
Exercise
What does backpropagation actually compute?
Need a hint?
It's an application of one rule from calculus.
Exercise
Why not initialize all weights to zero?
Need a hint?
Think about what gradients each hidden neuron would receive.
Exercise
With inverted dropout at rate p = 0.5, what happens to the surviving activations during TRAINING, and why?
Need a hint?
The expected activation seen by the next layer should not depend on whether dropout is on.
Exercise
Batch normalization and He initialization both help deep nets train. Which slot of the ML project loop do they modify — and does batch norm change what the network can represent?
Need a hint?
They make the parameter search easier; they don't add new functions to the model family.
Exercise
Transfer test — a technique this course did NOT teach: retrieval-augmented generation (RAG), which fetches documents and adds them to a model's prompt. In the agentic project loop, which slot does RAG change?
Need a hint?
It changes what information the model has access to at inference — its context.