What is a Vector?
The building block: a list of numbers with meaning.
On this page
Beginner: a vector is just an ordered list of numbers. That's it. The "ordered" part matters — [3, 5] is not the same as [5, 3], because each position stands for something specific (like "age" in position 1 and "income" in position 2). You can also picture a vector as an arrow starting at the origin and pointing somewhere in space — the numbers are the instructions for how far to walk along each axis.
Intermediate: there are two equally valid ways to think about a vector, and switching between them is a skill you'll use constantly: as a point (a location in space, useful when thinking about data) and as an arrow (a displacement with a direction and a magnitude, useful when thinking about motion, force, or change). A GPS coordinate is a point. "3km northeast" is an arrow. Both are described by the same two numbers.
Advanced: the number of entries in the list is called its dimensionality. A 2D vector lives on a page; a 3D vector lives in physical space; but nothing stops a vector from having 768 dimensions (a typical sentence embedding) or 4096 dimensions (a typical image embedding from a vision model) — you just can't draw it anymore, only compute with it. The math doesn't care how many dimensions there are; only your intuition needs the 2D/3D training wheels. Formally, the set of all n-dimensional real vectors forms a vector space, ℝⁿ — a structure closed under addition and scalar multiplication, which is the entire subject of section 1.9.
Read as: "v is a list of n numbers, living in n-dimensional real-number space." Some texts write vectors as columns instead of rows — — it's the exact same object, just laid out vertically; ML code (NumPy, PyTorch) treats both shapes as meaningfully different for matrix multiplication, so watch for it later.
Define the standard basis vectors e₁ = [1,0,…,0], e₂ = [0,1,…,0], …, eₙ = [0,0,…,1] — each a 1 in one position, zeros elsewhere.
Claim: any vector v = [v₁, v₂, …, vₙ] can be written as v = v₁e₁ + v₂e₂ + ⋯ + vₙeₙ, and this representation is unique.
Existence: compute the right-hand side component by component. The i-th component of v₁e₁ + ⋯ + vₙeₙ is vᵢ · 1 = vᵢ (every other term contributes 0 in that position, since eⱼ is zero everywhere except position j). This matches v exactly in every position, so the sum equals v.
Uniqueness: suppose also v = c₁e₁ + ⋯ + cₙeₙ for some other coefficients. Looking at component i of both sides gives vᵢ = cᵢ for every i — so the coefficients must be exactly v's own components. No other combination works.
Where this is used: this is the formal justification for treating "the vector" and "its list of coordinates" as interchangeable — every basis (not just the standard one) gives a similarly unique representation, which is exactly the property section 1.14 (Gram-Schmidt) and PCA (section 1.10) both rely on when changing to a more useful basis.
The vector [4, 3]: 4 steps right, 3 steps up from the origin. Drag the dot yourself once it settles.
Every ML framework (NumPy, PyTorch, TensorFlow) represents a vector as a 1-D array. Indexing, slicing, and elementwise math all "just work" the way you'd hope — this is precisely why learning the math pays off immediately in code.
import numpy as np
# A vector IS a 1-D NumPy array
house = np.array([1500, 3, 10]) # [sq ft, bedrooms, age]
velocity = np.array([60, 0]) # 60 km/h North
print(house.shape) # (3,) <- dimensionality
print(house[1]) # 3 <- number of bedrooms
print(house * 2) # scale every entry: [3000, 6, 20]- Tabular data — a house-price dataset describes each house as a vector:
[1500, 3, 10]could mean 1500 sq ft, 3 bedrooms, 10 years old. Every row of a spreadsheet is a vector. - Recommendation systems — a movie can be described as
[0.9, 0.1, 0.0]for how much Action/Comedy/Romance it contains; a user's taste can be described the same way, so "similar taste" becomes "nearby vectors." - Physics & robotics — velocity ("60 km/h North") is a vector because it has size and direction, unlike speed alone. A robot arm's joint angles at any instant are a vector describing its full pose.
- NLP embeddings — the word "king" might become a 300-dimensional vector such that
king − man + woman ≈ queen, purely from vector arithmetic. This is the basis of every modern language model's input layer. - Computer vision — a flattened 28×28 grayscale image is just a 784-dimensional vector before any neural network ever touches it.
- Confusing a vector's length (number of entries) with its norm (geometric size, see 1.8) — "length" in ML almost always means dimensionality, not magnitude.
- Assuming a 1-D NumPy array is automatically a row or column vector — it's neither, and this causes silent shape-mismatch bugs the moment you multiply it against a matrix. Reshape explicitly with
.reshape(-1, 1)when it matters.
Going deeper
In deep learning, vectors are usually called embeddings — dense lists of numbers (often 100–4000 dimensions) learned so that similar things end up as nearby vectors. A vector generalizes to a matrix (2D grid) and then a tensor (n-D grid) — the core data structure in PyTorch/TensorFlow.
One subtlety worth internalizing early: a vector by itself has no fixed "orientation" in memory — whether you treat it as a row or a column is a choice you make based on what operation comes next. Frameworks like NumPy default to 1-D arrays that are neither, and will silently broadcast in ways that surprise beginners. When in doubt, print .shape before trusting the math.
At the master level: the choice of basis (section 1.9) used to represent a vector as coordinates is arbitrary — the vector itself is a basis-independent geometric object. Changing basis (a "change-of-basis matrix") re-expresses the exact same vector with different numbers, which is precisely what happens internally during PCA and whitening transformations.
A vector = an ordered list of numbers = a point or arrow in space. Every row of your dataset is a vector.