Eigenvalues & Eigenvectors
The special directions a transformation doesn't rotate — only stretches.
On this page
Beginner: when a matrix transforms space, most vectors get both rotated and stretched. A few special vectors only get stretched or shrunk — their direction never changes. These are eigenvectors; the amount they scale by is the eigenvalue.
Intermediate: "eigen" is German for "own" or "characteristic" — an eigenvector is a direction that belongs to the matrix itself, independent of any particular input data. Once you know a matrix's eigenvectors and eigenvalues, you know almost everything important about how it behaves under repeated application.
Advanced: applying the matrix 100 times in a row is the same as scaling along each eigenvector by its eigenvalue raised to the 100th power — which is exactly how you'd analyze whether a dynamical system is stable, growing, or decaying over time (eigenvalues with magnitude > 1 mean growth along that direction; < 1 means decay). This single idea underlies stability analysis across control theory, ecology, economics, and recurrent neural networks.
Applying A to v gives back the same vector v, just scaled by λ. An n×n matrix has at most n distinct eigenvalues, found by solving det(A − λI) = 0 — a polynomial equation in λ called the characteristic equation.
Start directly from the definition and move everything to one side:
(inserting the identity matrix, section 1.6, so both terms are genuine matrix-vector products that can be factored together). This says v is in the null space of A − λI — but eigenvectors are required to be nonzero by definition. A matrix has a nonzero vector in its null space exactly when it's singular (section 1.9, 1.12), i.e. exactly when its determinant is zero:
For a 2×2 matrix A = [[a,b],[c,d]], expanding this determinant gives a concrete quadratic in λ:
solvable directly by the quadratic formula — for the worked example below (a=2, b=c=0, d=3), this becomes λ² − 5λ + 6 = 0 = (λ−2)(λ−3), giving exactly λ=2 and λ=3.
Where this is used: in practice, no library solves this polynomial equation for matrices larger than 2×2 or 3×3 — it's numerically unreliable at scale. Real eigensolvers use the QR algorithm (section 1.14) or power iteration (section 1.24) instead; this derivation is the "why it's true," not the "how it's computed."
For A = [[2,0],[0,3]], try v=[1,0]:
v=[1,0] is an eigenvector with eigenvalue λ=2. Similarly [0,1] has eigenvalue 3. For a diagonal matrix like this one, the eigenvalues are always just the diagonal entries themselves, and the eigenvectors are always the standard basis directions — no equation-solving required.
Drag the sliders — every grey arrow rotates and stretches under the matrix, but the two colored eigenvectors only ever change length, never direction.
np.linalg.eig returns eigenvalues and eigenvectors together — the columns of the second array, not the rows, are the eigenvectors, which trips up nearly everyone the first time.
import numpy as np
A = np.array([[2, 1], [1, 2]])
eigenvalues, eigenvectors = np.linalg.eig(A)
print(eigenvalues) # [3. 1.]
print(eigenvectors) # columns are the eigenvectors
# Verify: A @ v should equal lambda * v for each eigenpair
for i in range(len(eigenvalues)):
v = eigenvectors[:, i]
lam = eigenvalues[i]
print(np.allclose(A @ v, lam * v)) # True, True- PCA works by finding the eigenvectors of your data's covariance matrix — the directions of maximum variance. This is how you compress 100 correlated features (or a 10,000-pixel face image, "Eigenfaces") down to a handful of numbers. When you also have class labels and want directions that separate categories rather than just spread out variance, that's the generalized eigenvalue problem of section 1.29 (LDA).
- PageRank is, at its heart, finding the eigenvector of the web's link matrix with eigenvalue 1 — that eigenvector's entries become each page's importance score.
- Vibration analysis in mechanical engineering: the eigenvalues of a structure's stiffness matrix are literally its natural resonant frequencies.
- Markov chains — the long-run "steady state" of a random process is the eigenvector of the transition matrix with eigenvalue 1.
- Reading eigenvectors out of the rows of NumPy's output instead of the columns — a classic, silent bug.
- Assuming every real matrix has real eigenvalues — pure rotation matrices, for example, have no real eigenvectors at all (see the expert note below).
Going deeper
SVD (section 1.10) generalizes eigen-decomposition to non-square matrices and underlies matrix-factorization recommenders and NLP topic models.
Eigenvalues can be complex numbers even when the matrix is entirely real — this happens whenever the transformation includes genuine rotation with no fixed axis (a pure 2D rotation matrix, for instance, has no real eigenvectors at all, since every vector's direction changes). When people say "this matrix has no eigenvectors," they usually mean "no real ones" — complex eigenvectors always exist for any square matrix, over the complex numbers.
At the master level: not every square matrix can be fully diagonalized (written as A = PDP⁻¹ with D diagonal) — matrices with "repeated" eigenvalues but too few independent eigenvectors require the more general Jordan normal form. In practice, this is rarely an issue for the symmetric matrices (like covariance matrices) that dominate ML — the spectral theorem guarantees every real symmetric matrix diagonalizes cleanly with orthogonal eigenvectors, which is precisely why PCA always works without exception. Symmetric matrices whose eigenvalues are all strictly positive are exactly the positive definite matrices of section 1.15 — the two topics are two views of the same object.