Computing Eigenvalues & EigenvectorsOpen in Colab

Computing Eigenvalues & Eigenvectors

An eigenvector v\mathbf{v} of A\mathbf{A} satisfies Av=λv\mathbf{A}\mathbf{v} = \lambda\mathbf{v} — the matrix only scales it, never rotates it. Finding these special directions is a two-phase procedure: find the eigenvalues first from the characteristic polynomial, then find the eigenvectors by solving a linear system for each eigenvalue.

Finding the eigenvalues (the characteristic equation)

Phase 1 of the procedure: solve for λ\lambda from a determinant condition.

Rearrange Av=λv\mathbf{A}\mathbf{v} = \lambda\mathbf{v} as (A−λI)v=0(\mathbf{A} - \lambda\mathbf{I})\mathbf{v} = \mathbf{0}. For this to have a non-zero solution v\mathbf{v}, the matrix (A−λI)(\mathbf{A} - \lambda\mathbf{I}) must be singular (not invertible), which means its determinant is zero:

det⁡(A−λI)=0\det(\mathbf{A} - \lambda \mathbf{I}) = 0

For a 2×22\times2 matrix this gives a quadratic in λ\lambda (the characteristic polynomial); for n×nn\times n it gives a degree-nn polynomial.

Finding the eigenvectors for each eigenvalue

Phase 2 of the procedure: solve a homogeneous linear system for each λ\lambda.

Once we have a specific eigenvalue λk\lambda_k, substitute it back into (A−λkI)v=0(\mathbf{A} - \lambda_k\mathbf{I})\mathbf{v} = \mathbf{0} and solve. The solution is the null space (kernel) of (A−λkI)(\mathbf{A}-\lambda_k\mathbf{I}) — a whole line of directions (any scalar multiple of the eigenvector is also an eigenvector).

A full 2×2 example end-to-end (A=[4123]\mathbf{A} = \begin{bmatrix}4&1\\2&3\end{bmatrix})

Step 1: set up the characteristic equation

det⁡(A−λI)=det⁡[4−λ123−λ]=(4−λ)(3−λ)−(2)(1)=0\det(\mathbf{A} - \lambda\mathbf{I}) = \det\begin{bmatrix}4-\lambda & 1 \\ 2 & 3-\lambda\end{bmatrix} = (4-\lambda)(3-\lambda) - (2)(1) = 0

Expand:

12−4λ−3λ+λ2−2=λ2−7λ+10=012 - 4\lambda - 3\lambda + \lambda^2 - 2 = \lambda^2 - 7\lambda + 10 = 0

Step 2: solve for the eigenvalues

Factor: (λ−5)(λ−2)=0(\lambda - 5)(\lambda - 2) = 0, giving:

λ1=5,λ2=2\lambda_1 = 5, \qquad \lambda_2 = 2

Check via trace and determinant:

  • tr(A)=4+3=7=λ1+λ2=5+2\text{tr}(\mathbf{A}) = 4 + 3 = 7 = \lambda_1 + \lambda_2 = 5 + 2 ✓
  • det⁡(A)=4(3)−1(2)=10=λ1⋅λ2=5⋅2\det(\mathbf{A}) = 4(3) - 1(2) = 10 = \lambda_1 \cdot \lambda_2 = 5 \cdot 2 ✓

Step 3: eigenvector for λ1=5\lambda_1 = 5

Substitute into (A−5I)v=0(\mathbf{A} - 5\mathbf{I})\mathbf{v} = \mathbf{0}:

[4−5123−5][v1v2]=[−112−2][v1v2]=0\begin{bmatrix}4-5 & 1 \\ 2 & 3-5\end{bmatrix}\begin{bmatrix}v_1\\v_2\end{bmatrix} = \begin{bmatrix}-1 & 1 \\ 2 & -2\end{bmatrix}\begin{bmatrix}v_1\\v_2\end{bmatrix} = \mathbf{0}

Both rows encode the same constraint. Top row: −v1+v2=0⇒v2=v1-v_1 + v_2 = 0 \Rightarrow v_2 = v_1.

v1=[11](or any scalar multiple)\mathbf{v}_1 = \begin{bmatrix}1\\1\end{bmatrix} \quad \text{(or any scalar multiple)}

Verify: Av1=[4(1)+1(1)2(1)+3(1)]=[55]=5[11]\mathbf{A}\mathbf{v}_1 = \begin{bmatrix}4(1)+1(1)\\2(1)+3(1)\end{bmatrix} = \begin{bmatrix}5\\5\end{bmatrix} = 5\begin{bmatrix}1\\1\end{bmatrix} ✓

Step 4: eigenvector for λ2=2\lambda_2 = 2

Substitute into (A−2I)v=0(\mathbf{A} - 2\mathbf{I})\mathbf{v} = \mathbf{0}:

[2121][v1v2]=0\begin{bmatrix}2 & 1 \\ 2 & 1\end{bmatrix}\begin{bmatrix}v_1\\v_2\end{bmatrix} = \mathbf{0}

Both rows: 2v1+v2=0⇒v2=−2v12v_1 + v_2 = 0 \Rightarrow v_2 = -2v_1.

v2=[1−2]\mathbf{v}_2 = \begin{bmatrix}1\\-2\end{bmatrix}

Verify: Av2=[4(1)+1(−2)2(1)+3(−2)]=[2−4]=2[1−2]\mathbf{A}\mathbf{v}_2 = \begin{bmatrix}4(1)+1(-2)\\2(1)+3(-2)\end{bmatrix} = \begin{bmatrix}2\\-4\end{bmatrix} = 2\begin{bmatrix}1\\-2\end{bmatrix} ✓

Why both rows collapse to one constraint

When λ\lambda is an eigenvalue, (A−λI)(\mathbf{A}-\lambda\mathbf{I}) is singular by construction (det⁡=0\det = 0). A singular 2×22\times2 matrix has rank at most 1 — its two rows are proportional. That redundancy is exactly why infinitely many solutions exist (a whole line, i.e. the eigenvector direction).

Python
import numpy as np

A = np.array([[4., 1.], [2., 3.]])
eigenvalues, eigenvectors = np.linalg.eig(A)

print("Eigenvalues:", eigenvalues.round(3))   # [5., 2.]
print("Eigenvectors (columns):\n", eigenvectors.round(3))

# Verify Av = λv for each pair
for i in range(2):
    v   = eigenvectors[:, i]
    lam = eigenvalues[i]
    lhs = A @ v
    rhs = lam * v
    print(f"\nλ={lam:.0f}: Av={lhs.round(3)}, λv={rhs.round(3)}, match={np.allclose(lhs, rhs)}")

The eigendecomposition and easy matrix powers

With eigenvectors as columns V=[v1∣v2]\mathbf{V} = [\mathbf{v}_1 | \mathbf{v}_2] and eigenvalues on the diagonal:

A V=V Λ  ⟹  A=V Λ V−1\mathbf{A}\,\mathbf{V} = \mathbf{V}\,\boldsymbol{\Lambda} \;\Longrightarrow\; \mathbf{A} = \mathbf{V}\,\boldsymbol{\Lambda}\,\mathbf{V}^{-1}

For our example:

V=[111−2],Λ=[5002],V−1=1−3[−2−1−11]\mathbf{V} = \begin{bmatrix}1 & 1 \\ 1 & -2\end{bmatrix},\quad \boldsymbol{\Lambda} = \begin{bmatrix}5 & 0 \\ 0 & 2\end{bmatrix},\quad \mathbf{V}^{-1} = \frac{1}{-3}\begin{bmatrix}-2 & -1 \\ -1 & 1\end{bmatrix}

Power of A\mathbf{A}. The eigendecomposition makes powers trivial: An=VΛnV−1\mathbf{A}^n = \mathbf{V}\boldsymbol{\Lambda}^n\mathbf{V}^{-1}, and Λn=diag(5n,2n)\boldsymbol{\Lambda}^n = \mathrm{diag}(5^n, 2^n). As nn grows, the 5n5^n term dominates — the dominant eigenvector (1,1)(1,1) captures the "direction" of An\mathbf{A}^n for large nn.

Python
# Eigendecomposition and power
V    = eigenvectors
Lam  = np.diag(eigenvalues)
V_inv = np.linalg.inv(V)

A_reconstructed = V @ Lam @ V_inv
print("Reconstructed A:\n", A_reconstructed.round(3))  # should match A

A_cubed_direct = np.linalg.matrix_power(A, 3)
A_cubed_eigen  = V @ np.diag(eigenvalues**3) @ V_inv
print("A³ direct:\n", A_cubed_direct)
print("A³ via eigen:\n", A_cubed_eigen.round(3))

Power iteration for the dominant eigenvector

For large matrices where we only need the top eigenvector, power iteration is much cheaper than the full eigendecomposition:

Python
def power_iteration(A, n_iter=100, tol=1e-10):
    b = np.random.randn(A.shape[1])
    b /= np.linalg.norm(b)
    for _ in range(n_iter):
        b_new = A @ b
        lam   = np.dot(b, b_new)          # Rayleigh quotient: b^T A b / (b^T b) = b^T (Ab), since ||b||=1
        b_new /= np.linalg.norm(b_new)
        if np.linalg.norm(b_new - b) < tol:
            break
        b = b_new
    return lam, b

lam1, v1 = power_iteration(A)
print(f"Dominant eigenvalue: {lam1:.4f} (true: 5)")
print(f"Dominant eigenvector: {(v1/v1[0]).round(4)} (true: [1, 1])")

Power iteration converges at rate ∣λ2/λ1∣k|\lambda_2/\lambda_1|^k — the larger the gap between eigenvalues, the faster it converges.

Watch it run

Stepped on the same A\mathbf{A} worked by hand above, so it converges on the λ1=5\lambda_1 = 5, v1=(1,1)\mathbf{v}_1 = (1,1) you just derived — then re-run twice with the eigenvectors held fixed and only the spectrum changed:

Power iteration — the dominant eigenvector by attrition

1/9

Power iteration on the page's own matrix A = [[4, 1], [2, 3]], whose eigenvalues we already know are 5 and 2 with eigenvectors (1, 1) and (1, −2). The whole algorithm is: multiply by A, normalise, repeat. Nothing is solved and no polynomial is factored — the matrix is simply applied over and over, and the dominant direction wins by attrition. Starting from b = (1, 0), which is nothing special; the dashed teal line is the answer we are heading for.

python
1def power_iteration(A, n_iter=100,
2 tol=1e-10):
3 b = random_unit_vector(A.shape[1])
4 for k in range(1, n_iter + 1):
5 Ab = A @ b
6 # Rayleigh quotient, ||b|| = 1
7 lam = b @ Ab
8 b_new = Ab / norm(Ab)
9 if norm(b_new - b) < tol:
10 return lam, b_new
11 b = b_new
12 return lam, b
state
A
4.0001.000
2.0003.000
b on the unit circle; teal dashed = v₁, grey dashed = v₂
-1.851.851.25-1.25v₁xy

Written in the eigenbasis, b₀ = ⅔(1, 1) + ⅓(1, −2). Every multiplication by A scales the first part by 5 and the second by 2, so their ratio grows by 5/2 each step. Normalising just keeps the vector on the circle.

Run on the same A = [[4, 1], [2, 3]] worked by hand above, so it converges on the λ₁ = 5, v₁ = (1, 1) you already derived. Watch the angle error: it is multiplied by the same constant every iteration, and that constant turns out to be exactly |λ₂/λ₁| = 0.4. The last three frames change only the spectrum — the eigenvectors are held fixed — to show what that constant buys you: at |λ₂/λ₁| = 0.9 the same accuracy costs 129 iterations instead of 15, and at 1.0 the iteration cycles forever while the listing's stopping test never notices.

What to notice:

  • Nothing is solved. No determinant, no polynomial, no null space — just multiply and normalise. The dominant direction wins by attrition.
  • The angle error is multiplied by a constant each iteration, and that constant measures out to exactly ∣λ2/λ1∣=0.4|\lambda_2/\lambda_1| = 0.4. This is the rate claim above, checked rather than asserted.
  • The estimate approaches 5 from above. For a symmetric matrix the Rayleigh quotient can never exceed λ1\lambda_1; this A\mathbf{A} is not symmetric, so it can and does.
  • Raising λ2\lambda_2 to 4.5 costs 129 iterations instead of 15 for the same accuracy, with the target eigenvector unchanged. The matrix does not look any harder — the eigenvalues are just closer together.
  • At λ2=−5\lambda_2 = -5 it never converges at all, falling into an exact period-2 cycle. Worse, the norm(b_new - b) < tol test above cannot detect that: consecutive iterates stay far apart, so it runs out the loop and returns a meaningless λ\lambda with no error. A convergence test that can only report success is not a convergence test — check the residual ∥Ab−λb∥\|\mathbf{Ab} - \lambda\mathbf{b}\| instead.

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
Computing Eigenvalues & Eigenvect…Vanishing & Exploding Gra…SVD and the Eckart-Young …Eigenvalues and Eigenvect…Principal Component Analy…