Quiz: GPU Programming
Six questions spanning the whole course — the SIMT execution model, CUDA indexing, memory coalescing and the roofline, GPUs for deep learning, and distributed training.
Exercise
What does SIMT (Single Instruction, Multiple Threads) mean for the 32 threads of a warp?
Need a hint?
One instruction stream, many data lanes.
Exercise
A vector-add kernel is launched over n elements that is not a multiple of the block size. Why must the kernel include `if (i < n)`?
Need a hint?
You can only launch a whole number of blocks.
Exercise
Two kernels do identical arithmetic, but kernel A has consecutive threads read consecutive addresses while kernel B has each thread read with a large stride. What is the likely outcome?
Need a hint?
Coalesced vs scattered global access.
Exercise
On a roofline plot, a large matrix multiply sits to the right of the ridge point (high arithmetic intensity). What does this tell you?
Need a hint?
Which ceiling — compute or memory — is binding?
Exercise
Why is generating one token at a time with batch size 1 (LLM decode) typically memory-bound, and what fixes it?
Need a hint?
How much arithmetic happens per byte of weights loaded?
Exercise
A model's parameters fit on one GPU, but its optimizer state does not. Which is the smallest change that fixes this without adding tensor or pipeline parallelism?
Need a hint?
Start from the lowest ZeRO stage and stop as soon as it fits.
Exercise
Memory coalescing, kernel fusion, and tensor-parallel sharding all make training run faster without changing the model or the loss. Which slot of the ML project loop do they serve?
Need a hint?
They make the parameter search physically feasible — the systems level of one slot.
Exercise
Transfer test — a technique this course did NOT teach: computing an ROC curve and its AUC to compare two classifiers. Which slot of the ML project loop does it belong to?
Need a hint?
It's a way to measure and compare model quality.