What is a Matrix?
A grid of numbers — and a machine for transforming vectors.
On this page
Beginner: a matrix is a rectangular grid of numbers, arranged in rows and columns — think spreadsheet. But it's also a transformation: a rule that takes any vector in and produces a new vector out (rotated, stretched, squashed, or flipped). Multiplying a vector by a matrix means "apply this transformation."
Intermediate: these two views — "a matrix is a table of data" and "a matrix is a function that moves vectors around" — feel unrelated at first, but they're the same object seen from two angles. When a matrix stores a dataset, its rows are individual data points. When that same matrix multiplies a vector, its columns tell you where each input axis ends up after the transformation.
Advanced: this is exactly why neural network weight matrices are matrices — they're literally the transformation each layer applies to its input. Stack several such transformations (with a non-linear function between each) and you get a deep network: a chain of linear-algebra operations punctuated by small non-linear "kinks" that let the whole chain approximate functions no single matrix could represent alone.
The subscript a_ij means "the entry in row i, column j" — always row first, then column. A matrix with the same number of rows and columns is square; otherwise it's rectangular. Both are common in ML: square matrices show up in transformations and covariance; rectangular matrices show up almost everywhere else (a dataset is rarely square).
Claim: for any matrix A and vectors x, y and scalar c, A(x+y) = Ax + Ay and A(cx) = c(Ax) — the two properties that define a linear map, which is exactly why "matrix" and "linear transformation" are treated as the same idea throughout this chapter.
Proof: row i of A(x+y) is the dot product of A's i-th row with x+y:
— using ordinary distributivity of multiplication over addition, term by term. The first sum is exactly row i of Ax, and the second is row i of Ay. Since this holds for every row i, A(x+y) = Ax + Ay. The scalar case is even shorter: row i of A(cx) is Σⱼ aᵢⱼ(cxⱼ) = c Σⱼ aᵢⱼxⱼ, which is exactly c times row i of Ax.
Where this is used: linearity is precisely what makes a neural network layer (section 1.5) predictable and differentiable in closed form — every "linear layer" in every framework is called that specifically because it satisfies this theorem.
Click any cell to see its row (a student's full record) and column (everyone's score in one subject) highlighted together.
arr[row, :] pulls a full row, arr[:, col] pulls a full column — the exact row/column duality shown in the diagram above.
#include <cstdio>
#include <utility>
#include <vector>
using Matrix = std::vector<std::vector<double>>;
std::pair<size_t, size_t> shape(const Matrix& m) {
return {m.size(), m.empty() ? 0 : m[0].size()};
}
std::vector<double> getRow(const Matrix& m, size_t i) { return m[i]; }
std::vector<double> getCol(const Matrix& m, size_t j) {
std::vector<double> col;
for (const auto& row : m) col.push_back(row[j]);
return col;
}
Matrix transpose(const Matrix& m) {
size_t rows = m.size(), cols = m[0].size();
Matrix t(cols, std::vector<double>(rows));
for (size_t r = 0; r < rows; ++r)
for (size_t c = 0; c < cols; ++c)
t[c][r] = m[r][c];
return t;
}
int main() {
Matrix scores = {
{72, 3, 1},
{65, 2, 0},
{88, 4, 1},
};
auto s = shape(scores);
std::printf("(%zu, %zu)\n", s.first, s.second); // (3, 3)
auto row1 = getRow(scores, 1);
std::printf("[%.0f, %.0f, %.0f]\n", row1[0], row1[1], row1[2]); // [65, 2, 0]
auto col0 = getCol(scores, 0);
std::printf("[%.0f, %.0f, %.0f]\n", col0[0], col0[1], col0[2]); // [72, 65, 88]
auto t = transpose(scores);
auto ts = shape(t);
std::printf("(%zu, %zu)\n", ts.first, ts.second); // still (3, 3)
return 0;
}- A spreadsheet of 5 students' scores across 3 subjects is a 5×3 matrix — each row is a student (a vector!), each column a subject.
- A grayscale photo is literally a matrix — each cell is a pixel's brightness (0=black, 255=white). A 1080p photo is a 1080×1920 matrix of numbers, nothing more exotic than that.
- A user-movie ratings table (Netflix, Spotify) is a matrix where rows are users, columns are items, and most cells are empty — this is exactly the matrix that recommender systems try to "fill in."
- An adjacency matrix represents a graph or social network: entry (i, j) is 1 if node i connects to node j, and 0 otherwise — this is how graph neural networks represent structure numerically.
- Mixing up row-major vs. column-major mental models when reading
a_ijnotation — it is always row, then column, in every standard ML text and library. - Assuming "matrix" always means "square" — most real datasets are rectangular, and many matrix operations (like the inverse) simply don't apply to non-square matrices at all.
Going deeper
A color image adds a channel dimension (R/G/B), making it a tensor of shape (height × width × 3). A batch of images in a CNN is a 4D tensor: (batch × height × width × channels).
Framework conventions differ here and cause real bugs: PyTorch defaults to (batch, channels, height, width) ("channels-first"), while TensorFlow/Keras and most image files default to (batch, height, width, channels) ("channels-last"). Mixing them up silently produces garbage results rather than an error, since both are just 4D tensors of numbers — always check a library's expected shape convention before feeding it data.
At the master level: any matrix can be viewed as a linear map between two vector spaces, ℝⁿ → ℝᵐ. This abstraction is what lets the exact same theory (rank, span, eigenvalues, SVD) apply identically whether the matrix represents a dataset, a neural network layer, a rotation in 3D graphics, or a quantum operator — it's the same mathematical object underneath every one of those applications.