Introduction
How this chapter is structured, the notation it uses, and where to start.
On this page
This chapter builds the linear algebra behind machine learning from the ground up — starting with a single arrow on a page and ending, 34 lessons later, with the tools behind PageRank, kernel SVMs, conjugate gradient solvers, and normalizing-flow generative models. Every idea is built on the ones before it: nothing here is presented as a rule to memorize without first showing why it has to be true.
It's written for three overlapping audiences at once — a beginner meeting vectors and matrices for the first time, an intermediate reader who's used these tools but wants the "why" behind them, and an advanced reader who wants the actual theorems, proofs, and numerical caveats that production ML code quietly depends on. Every lesson is written in all three registers, so you can read at whichever level matches a given topic for you and go deeper exactly where you need to.
Each of the 34 lessons that follow uses the same consistent shape, so you always know where to look for what you need:
- In plain English — the idea explained three times, at increasing depth (beginner → intermediate → advanced), with no formulas required to follow along.
- Formula — the precise mathematical statement, in standard notation.
- Derivation — a full proof or worked derivation of the formula, plus a "where this is used" note connecting it to a real ML system.
- Diagram — most lessons include a live, draggable diagram you can manipulate directly, rather than a static picture.
- Practical example — runnable NumPy/PyTorch/scikit-learn code showing the idea used the way you'd actually use it.
- Real-world examples, common mistakes, and going deeper — where the idea shows up in production systems, what trips people up in practice, and an optional deeper dive for readers who want more.
A small, consistent set of conventions is used in every lesson:
- Lowercase letters (
x,v) are vectors; uppercase letters (A,B) are matrices; plain lowercase italics (a,λ) are scalars. - is the transpose (section 1.6), is the inverse (also section 1.6), and
Aᵢⱼmeans the entry in row i, column j. - is a norm (section 1.8, length/size of a vector or matrix), and
v·wis the dot product (section 1.3). - is the gradient of a scalar function (section 1.17), and outside a matrix name means summation; as a matrix name (as in
Σ = VΛVᵀ) it means something specific to that lesson and is always defined on the spot.
Comfort with basic algebra (solving for x, working with exponents) and functions is all that's assumed going in. A little single-variable calculus (what a derivative means) helps for the "Advanced" paragraphs and the "Going deeper" boxes in the later lessons, but is never required to follow a lesson's main thread — every calculus-dependent idea is explained again from scratch the first time it's needed.
Start with section 1.2, What is a Vector?, and move through the chapter in order — each lesson builds directly on the ones before it, all the way to the final lesson on normalizing flows, which uses ideas from nearly every earlier section at once.