The Directional Derivative
A partial derivative answers a narrow question: how fast does change if I nudge one coordinate, holding the rest fixed? The directional derivative answers the general one: how fast does change if I step in an arbitrary direction ? Partial derivatives are just the special case where is a coordinate axis. This single quantity is what makes "the gradient points in the direction of steepest ascent" a theorem rather than a slogan.
The limit definition of the directional derivative
Fix a point and a unit direction (). Slide along the ray and measure the per-unit-distance change:
Setting (the -th standard basis vector) recovers the partial derivative exactly, so partials are directional derivatives along the axes.
The unit-vector requirement is not optional
is a rate per unit distance. If is not normalized, the limit scales by and you get a rate per unit of instead — an apples-to-oranges quantity. Always divide by first: .
Reduction to the dot product with the gradient
Define the single-variable restriction . By the limit definition, . Now differentiate with the multivariable chain rule. Writing the path as , we have , so:
Evaluating at gives the central result:
The directional derivative in any direction is just the projection of the gradient onto that direction. The gradient packages every directional derivative at once — compute it once, dot it with any you like.
Why the gradient points toward steepest ascent
Expand the dot product geometrically. With the angle between and , and :
Since ranges over , the directional derivative is bounded:
| Direction | Interpretation | ||
|---|---|---|---|
| Along | steepest ascent | ||
| Against | steepest descent | ||
| Perpendicular to | along a level set (contour): unchanged |
The maximum is attained when , i.e. . This proves that the gradient points in the direction of fastest increase, and its magnitude is that fastest rate. Gradient descent takes the opposite direction precisely because minimizes .
Gradients are orthogonal to level sets
The third row explains why is always perpendicular to the contour through : moving along a contour leaves unchanged, so the directional derivative there is , which forces .
A concrete numerical example
Let at the point .
Gradient:
Directional derivative toward . First normalize. This is the step beginners skip:
Sanity check against the extremes. The steepest possible rate here is . Our value , as it must be — is close to but not exactly aligned with .
import numpy as np
def f(x, y):
return x**2 + 3*x*y
def grad_f(x, y):
return np.array([2*x + 3*y, 3*x])
p = (1.0, 2.0)
g = grad_f(*p) # [8., 3.]
v = np.array([3.0, 4.0])
u = v / np.linalg.norm(v) # unit vector [0.6, 0.8]
Du = g @ u # directional derivative
print(f"gradient ∇f = {g}")
print(f"unit direction û = {u}")
print(f"D_û f = {Du:.3f}") # 7.200
print(f"steepest rate = {np.linalg.norm(g):.3f}") # 8.544 (>= 7.200, as required)
# Finite-difference check of the limit definition
h = 1e-6
fd = (f(*(np.array(p) + h*u)) - f(*p)) / h
print(f"finite-diff = {fd:.3f}") # ≈ 7.200
Common pitfalls
- Forgetting to normalize . Using a non-unit vector inflates the result by . In the example, skipping normalization gives instead of .
- Assuming partials existing implies the directional derivative exists in every direction. For non-differentiable functions (e.g. at the origin) the chain-rule shortcut can fail; you must fall back to the limit definition. The dot-product formula is guaranteed only where is differentiable.
- Treating as a vector. It is a scalar — a rate. The direction of fastest change is the vector .
Related concepts
- Derivatives and Gradients: partial derivatives and the gradient vector this page builds on
- Chain Rule and Backpropagation: the multivariable chain rule used to derive the dot-product formula
- Multivariable Optimization: where steepest descent and curvature drive optimization