Computing Eigenvalues & Eigenvectors
An eigenvector of satisfies — the matrix only scales it, never rotates it. Finding these special directions is a two-phase procedure: find the eigenvalues first from the characteristic polynomial, then find the eigenvectors by solving a linear system for each eigenvalue.
Finding the eigenvalues (the characteristic equation)
Phase 1 of the procedure: solve for from a determinant condition.
Rearrange as . For this to have a non-zero solution , the matrix must be singular (not invertible), which means its determinant is zero:
For a matrix this gives a quadratic in (the characteristic polynomial); for it gives a degree- polynomial.
Finding the eigenvectors for each eigenvalue
Phase 2 of the procedure: solve a homogeneous linear system for each .
Once we have a specific eigenvalue , substitute it back into and solve. The solution is the null space (kernel) of — a whole line of directions (any scalar multiple of the eigenvector is also an eigenvector).
A full 2×2 example end-to-end ()
Step 1: set up the characteristic equation
Expand:
Step 2: solve for the eigenvalues
Factor: , giving:
Check via trace and determinant:
- ✓
- ✓
Step 3: eigenvector for
Substitute into :
Both rows encode the same constraint. Top row: .
Verify: ✓
Step 4: eigenvector for
Substitute into :
Both rows: .
Verify: ✓
Why both rows collapse to one constraint
When is an eigenvalue, is singular by construction (). A singular matrix has rank at most 1 — its two rows are proportional. That redundancy is exactly why infinitely many solutions exist (a whole line, i.e. the eigenvector direction).
import numpy as np
A = np.array([[4., 1.], [2., 3.]])
eigenvalues, eigenvectors = np.linalg.eig(A)
print("Eigenvalues:", eigenvalues.round(3)) # [5., 2.]
print("Eigenvectors (columns):\n", eigenvectors.round(3))
# Verify Av = λv for each pair
for i in range(2):
v = eigenvectors[:, i]
lam = eigenvalues[i]
lhs = A @ v
rhs = lam * v
print(f"\nλ={lam:.0f}: Av={lhs.round(3)}, λv={rhs.round(3)}, match={np.allclose(lhs, rhs)}")
The eigendecomposition and easy matrix powers
With eigenvectors as columns and eigenvalues on the diagonal:
For our example:
Power of . The eigendecomposition makes powers trivial: , and . As grows, the term dominates — the dominant eigenvector captures the "direction" of for large .
# Eigendecomposition and power
V = eigenvectors
Lam = np.diag(eigenvalues)
V_inv = np.linalg.inv(V)
A_reconstructed = V @ Lam @ V_inv
print("Reconstructed A:\n", A_reconstructed.round(3)) # should match A
A_cubed_direct = np.linalg.matrix_power(A, 3)
A_cubed_eigen = V @ np.diag(eigenvalues**3) @ V_inv
print("A³ direct:\n", A_cubed_direct)
print("A³ via eigen:\n", A_cubed_eigen.round(3))
Power iteration for the dominant eigenvector
For large matrices where we only need the top eigenvector, power iteration is much cheaper than the full eigendecomposition:
def power_iteration(A, n_iter=100, tol=1e-10):
b = np.random.randn(A.shape[1])
b /= np.linalg.norm(b)
for _ in range(n_iter):
b_new = A @ b
lam = np.dot(b, b_new) # Rayleigh quotient: b^T A b / (b^T b) = b^T (Ab), since ||b||=1
b_new /= np.linalg.norm(b_new)
if np.linalg.norm(b_new - b) < tol:
break
b = b_new
return lam, b
lam1, v1 = power_iteration(A)
print(f"Dominant eigenvalue: {lam1:.4f} (true: 5)")
print(f"Dominant eigenvector: {(v1/v1[0]).round(4)} (true: [1, 1])")
Power iteration converges at rate — the larger the gap between eigenvalues, the faster it converges.
Watch it run
Stepped on the same worked by hand above, so it converges on the , you just derived — then re-run twice with the eigenvectors held fixed and only the spectrum changed:
Power iteration — the dominant eigenvector by attrition
Power iteration on the page's own matrix A = [[4, 1], [2, 3]], whose eigenvalues we already know are 5 and 2 with eigenvectors (1, 1) and (1, −2). The whole algorithm is: multiply by A, normalise, repeat. Nothing is solved and no polynomial is factored — the matrix is simply applied over and over, and the dominant direction wins by attrition. Starting from b = (1, 0), which is nothing special; the dashed teal line is the answer we are heading for.
| 4.000 | 1.000 | |
| 2.000 | 3.000 |
Written in the eigenbasis, b₀ = ⅔(1, 1) + ⅓(1, −2). Every multiplication by A scales the first part by 5 and the second by 2, so their ratio grows by 5/2 each step. Normalising just keeps the vector on the circle.
Run on the same A = [[4, 1], [2, 3]] worked by hand above, so it converges on the λ₁ = 5, v₁ = (1, 1) you already derived. Watch the angle error: it is multiplied by the same constant every iteration, and that constant turns out to be exactly |λ₂/λ₁| = 0.4. The last three frames change only the spectrum — the eigenvectors are held fixed — to show what that constant buys you: at |λ₂/λ₁| = 0.9 the same accuracy costs 129 iterations instead of 15, and at 1.0 the iteration cycles forever while the listing's stopping test never notices.
What to notice:
- Nothing is solved. No determinant, no polynomial, no null space — just multiply and normalise. The dominant direction wins by attrition.
- The angle error is multiplied by a constant each iteration, and that constant measures out to exactly . This is the rate claim above, checked rather than asserted.
- The estimate approaches 5 from above. For a symmetric matrix the Rayleigh quotient can never exceed ; this is not symmetric, so it can and does.
- Raising to 4.5 costs 129 iterations instead of 15 for the same accuracy, with the target eigenvector unchanged. The matrix does not look any harder — the eigenvalues are just closer together.
- At it never converges at all, falling into an exact
period-2 cycle. Worse, the
norm(b_new - b) < toltest above cannot detect that: consecutive iterates stay far apart, so it runs out the loop and returns a meaningless with no error. A convergence test that can only report success is not a convergence test — check the residual instead.
Related concepts
- Eigenvalues and Eigenvectors: intuition, PCA connection, SVD generalization
- PCA & Dimensionality Reduction: eigenvectors of the covariance matrix
- Vanishing & Exploding Gradients: spectral radius is the dominant eigenvalue magnitude