The Orthogonal Procrustes Problem
The exact, closed-form way to align two shapes or embedding spaces via SVD.
On this page
Beginner: imagine you have two versions of the same shape — one rotated relative to the other — and you want to find the exact rotation that lines them up as closely as possible. The orthogonal Procrustes problem is precisely this: find the best rotation (or reflection) matrix that aligns one set of points to another, minimizing the total squared distance between corresponding points.
Intermediate: remarkably, this has an exact, closed-form answer, and it comes directly from the SVD (section 1.10): compute the cross-covariance matrix between the two point sets, take its SVD UΣVᵀ, and the optimal rotation is simply R = UVᵀ — no iterative optimization needed at all.
Advanced: this is the standard tool for embedding alignment — for instance, aligning word-embedding spaces trained independently on two different languages, so that a word and its translation end up at (approximately) the same point after applying the optimal rotation.
Minimizing ‖AR − B‖²_F over rotations R is equivalent to maximizing a simpler quantity. Expand the squared Frobenius norm using ‖X‖²_F = trace(XᵀX) (section 1.20):
The first term equals trace(AᵀA) by the cyclic property (section 1.23) since RᵀR = I, and the last term doesn't involve R at all — so minimizing the whole expression over R is exactly equivalent to maximizing trace(RᵀAᵀB) = trace(RᵀM) where M = AᵀB = UΣVᵀ. Substitute the SVD and use cyclic invariance again:
Z is a product of orthogonal matrices, so it's orthogonal too, meaning every entry of Z satisfies |Z_{ii}| ≤ 1. Since Σ has non-negative diagonal entries, trace(ZΣ) = Σᵢ Z_{ii}σᵢ is maximized exactly when every Z_{ii} = 1, i.e. when Z = I. Solve VᵀRᵀU = I for R: left-multiply by V to get RᵀU = V (using VVᵀ = I), then right-multiply by Uᵀ to get Rᵀ = VUᵀ (using UUᵀ = I). Transposing both sides gives R = UVᵀ — the closed form, derived entirely from properties of trace and orthogonal matrices already covered in this chapter, with no iterative search required.
Where this is used: every cross-lingual embedding alignment pipeline and shape-registration tool that calls this a "one-line SVD solution" is relying on exactly this proof — it's also why the reflection-vs-rotation subtlety noted below is unavoidable: the proof only ever concluded Z = I, not that det(R) = +1.
Drag to rotate the orange shape by hand, or let the closed-form SVD solution snap it into perfect alignment instantly.
Three lines after the SVD call, and the alignment is exact — this is the entire algorithm used in real embedding-alignment pipelines.
import numpy as np
target = np.random.randn(20, 2)
true_theta = 0.7
rot = np.array([[np.cos(true_theta), -np.sin(true_theta)],
[np.sin(true_theta), np.cos(true_theta)]])
source = target @ rot.T # a rotated copy of the same shape
# Solve for the rotation that undoes this, via SVD:
M = source.T @ target
U, S, Vt = np.linalg.svd(M)
R = U @ Vt # the optimal alignment rotation
aligned = source @ R
print(np.allclose(aligned, target, atol=1e-6)) # True -- perfect recovery- Cross-lingual word embeddings — aligning independently trained embedding spaces from two languages using a small bilingual dictionary as anchor points, then applying Procrustes to the rest of the vocabulary.
- Shape analysis and computer vision — comparing 3D scanned objects or anatomical landmarks that were captured at arbitrary orientations.
- Comparing neural network representations across different training runs or random seeds — Procrustes alignment is a standard tool for checking whether two networks learned "the same" internal representation, just rotated.
- Forgetting the two point sets must already be correctly matched (point i in set A corresponds to point i in set B) — Procrustes solves for the best rotation given a known correspondence, it does not discover the correspondence itself.
- Swapping the order to
R = VUᵀ— the correct formula depends on which matrix the cross-covariance is built from; withM = AᵀB = UΣVᵀas defined above the answer isR = UVᵀ, notVUᵀ(defining the cross-covariance the other way round,BᵀA, would swap U and V and flip which order is correct) — always double-check against a known test case.
Going deeper
A subtlety: the raw SVD solution can produce a reflection rather than a pure rotation if the determinant of UVᵀ comes out negative — the standard fix flips the sign of the last column of V (or the corresponding singular value) to force a proper rotation when one is specifically required.
At the master level: Procrustes analysis more generally also allows solving for an optimal scale factor and translation alongside the rotation (full "similarity transformation" Procrustes) — the rotation piece is unchanged, computed exactly as above, with scale and translation solved for separately in closed form once the optimal rotation is known.