Learning Paradigms
Supervised, unsupervised, semi-supervised, self-supervised & reinforcement learning.
On this page
Beginner: picture a folder of 10,000 house-listing photos. If every photo comes with a price tag stapled to it, you're in supervised learning — the goal is to learn a rule that predicts the tag from the photo. Take away every price tag and you're in unsupervised learning — there's no "right answer" to predict anymore, but you can still ask what the photos have in common: which ones look alike, which form natural groups. Semi-supervised learning is the realistic middle ground — a listings site might have confirmed sale prices for the 200 houses that already sold and no price at all for the other 9,800 that are merely listed, and you'd like to use all 10,000 photos, not just the 200. Self-supervised learning throws away price tags too, but instead of giving up on having a "right answer," it manufactures one out of the photo itself — black out the roof and ask the model to guess what's missing, or rotate the photo and ask which way is up. And reinforcement learning isn't a photo folder at all: think of a thermostat deciding when to turn the heat on, watching the room temperature and the energy bill react to its own past choices over time, with no dataset of "correct" actions ever handed to it up front.
Worked example — the same data, five different jobs: take a bank's 50,000 most recent transactions. Framed as supervised learning, you'd use only the roughly 500 of those that were actually escalated and investigated, each one carrying a confirmed "fraud" or "legitimate" tag, and train a model to predict that tag from the transaction's own features. Framed as unsupervised learning, you throw the tag away entirely — cluster all 50,000 by spending pattern, merchant category, and time of day, or fit a density model, and flag whichever transactions sit in the sparsest, most unusual region of that space, with no notion of "fraud" involved at all, only "unusual." Semi-supervised learning uses both pieces of the same 50,000 rows at once: the 500 confirmed tags anchor where the boundary should sit, and the other 49,500 unlabeled transactions get pulled toward whichever side of that boundary they cluster nearest to. Self-supervised learning ignores the fraud tag too, but manufactures its own training signal out of the untagged bulk — mask the transaction amount and train the model to predict it from the rest of the record — which builds a general sense of what a normal transaction looks like before the model has ever seen a single confirmed fraud case; that learned representation is then fine-tuned on the 500 tagged examples. And reinforcement learning abandons the fixed 50,000-row dataset altogether: a live fraud-review system decides, transaction by transaction, whether to approve, hold, or block it, and adjusts its policy based on the downstream cost of each choice — an angry customer from a wrongly blocked purchase, real money lost from an approved fraud — a loop that keeps running and adapting as fraud patterns shift, long after any fixed dataset would have gone stale. The rows never change; only which columns you're allowed to look at, and whether "the data" is even a fixed table at all, does.
Intermediate: supervised learning assumes you observe i.i.d. pairs (x, y) drawn from some unknown joint distribution, and the goal is to learn a function that generalizes the x → y mapping to new, unseen x. Unsupervised learning observes only the x's — there is no y at all — so the goal shifts to describing structure in that distribution itself: density, clusters, a lower-dimensional manifold the data actually lives on. Semi-supervised learning gets a large sample of x's and a much smaller sample of matched (x, y) pairs, and the unlabeled data only helps if it carries real information about where the decision boundary should sit — typically via a cluster assumption (points in the same dense region tend to share a label) or a manifold assumption (nearby points on the underlying data manifold share a label). Self-supervised learning is a trick for turning an unsupervised-looking pile of raw x's back into ordinary supervised pairs: a pretext task carves an (x, y) pair out of each unlabeled example automatically — mask a token and predict it, crop two views of an image and predict that they came from the same photo — then trains with a completely standard supervised loss. Reinforcement learning drops the fixed-dataset assumption altogether: an agent chooses actions inside an environment, receives a reward after each one (often delayed and sparse), and which states and actions it even gets to see depends on the very policy it's in the middle of learning — something none of the other four paradigms have to deal with.
Advanced: all five are answers to one underlying question — what information is available to estimate the risk in the formula below, and what can the loss actually see? Supervised learning evaluates the loss pointwise on a labeled pair. Unsupervised learning replaces the loss with something that only needs x — reconstruction error, negative log-likelihood under a density model, a clustering distortion objective — so "no labels" does not mean "no assumptions"; the assumptions simply moved into the choice of loss and hypothesis class instead of coming from human labels. Self-supervised learning is not a new kind of loss at all — it is ordinary supervised empirical risk minimization, with the one twist that the label-generating function was written by the practitioner instead of hired out to human annotators, which is exactly why self-supervised objectives look and behave like supervised ones under the hood. Reinforcement learning is the odd one out mathematically, not just practically: because the reward for an action often arrives many steps later, you cannot even write down a per-decision loss the way empirical risk minimization assumes — the object being optimized is an expected, discounted sum of rewards over an entire trajectory, and the genuinely hard problem — credit assignment, deciding which of the last fifty actions actually caused this reward — has no analogue in the other four paradigms at all.
Empirical risk minimization — choose the function f, out of some allowed hypothesis class ℋ, that minimizes the average loss over the data you have. Every paradigm above is this same minimization, with a different answer to "what data estimates the sum, and what does L measure." Supervised learning has direct (x, y) pairs, so L compares a prediction to a true label directly — squared error, cross-entropy. Unsupervised learning has only x's, so L has to be rewritten to not need y at all — reconstruction error, the negative log-likelihood of x under a fitted density, or a clustering distortion like the k-means objective used in the diagram below. Semi-supervised learning literally sums two losses over two different subsets of the same n — a supervised term over the few labeled points, plus an unsupervised term over the many unlabeled ones, weighted against each other. Self-supervised learning keeps L exactly as it is in the supervised case — it just generates y mechanically from x instead of collecting it from a human. Reinforcement learning replaces the whole framing: there is no fixed n and no per-example (x, y) pair at all — the object being maximized is an expected return accumulated over an entire sequence of decisions rather than compared against a single input-output pair, which is why its own empirical-risk view is deferred to a dedicated module later in this chapter instead of being forced into the formula above.
Define two risks for a candidate function f that is fixed — chosen in advance, not yet fit to any particular sample:
R(f) is the true risk — the average loss f would incur over the entire, unobservable population P. R̂(f) is the empirical risk — the average loss actually measured on the n training samples you happen to have drawn. Because those samples are i.i.d. draws from P and f is fixed, each term L(f(x_i), y_i) is an independent, identically distributed copy of the same random variable L(f(x), y). Linearity of expectation then gives:
So for any f fixed before this sample was drawn, the empirical risk is an unbiased estimator of the true risk, and by the law of large numbers it concentrates around R(f) more tightly as n grows. That sounds like it should make empirical risk minimization trustworthy by construction — it doesn't, and the reason is worth stating carefully.
The proof above requires f to be fixed before the sample is drawn. The f̂ that empirical risk minimization actually returns is not fixed in advance — it is chosen specifically because it looks good on this exact sample: f̂ = argmin over f in ℋ of R̂(f). That search preferentially selects whichever candidate happens to look best on this training set, which means it selects not only for genuinely low true risk, but partly for having drawn a favorable noise realization on this particular sample. Once f̂ depends on the same data used to compute R̂(f̂), the independence that made the expectation calculation above valid is gone — you can no longer swap "take an expectation" and "evaluate at f̂" the way the derivation did for a fixed f. In practice this shows up as:
known as the optimism of the training error: the empirical risk measured at the chosen solution systematically underestimates that same solution's true risk, on average. The size of this gap — R(f̂) − R̂(f̂), the generalization gap — grows with how large and flexible ℋ is, since a bigger hypothesis class simply gives the argmin more candidates to search through for a lucky-looking fit. That growth is exactly the seed of the overfitting and capacity discussion later in this module.
Where this is used: this is the entire reason held-out validation and test sets exist. Evaluate f̂'s loss on a fresh batch of samples that played no role in choosing f̂, and independence is restored — that fresh empirical risk is once again an unbiased estimate of R(f̂), exactly as the derivation above proves for any fixed f. Reusing training data to both fit a model and report its performance measures optimism, not accuracy.
Every dot is a simulated house at a fixed (square footage, distance from downtown) position — the positions never move across views. Only how much of each dot's true price bracket you're allowed to see changes as you switch paradigm; the slider controls exactly how many labels are revealed in the semi-supervised view.
The same ten (x1, x2) points are used for both halves below — the supervised half additionally reads their label; the unsupervised k-means half never touches the label field at all, and still finds essentially the same two groups just from geometry. The from-scratch columns show every gradient and every centroid update explicitly; the library column shows the same two computations the way you'd actually write them.
import math
import random
# The same 10 points, used two different ways.
# Each point is (feature_1, feature_2, true_label) — the label is only
# ever read by the supervised half below; the unsupervised half never
# looks at the third element at all.
points = [
(1.0, 2.0, 0), (1.2, 1.8, 0), (1.5, 2.2, 0), (1.1, 2.5, 0), (1.4, 1.9, 0),
(3.0, 4.0, 1), (3.2, 3.8, 1), (3.5, 4.2, 1), (3.1, 4.5, 1), (3.4, 3.9, 1),
]
# --- (a) SUPERVISED: empirical risk minimization for logistic regression ---
# Hypothesis: f(x) = sigmoid(w1*x1 + w2*x2 + b)
# Loss: binary cross-entropy, averaged over all n=10 labeled points.
def sigmoid(z):
return 1.0 / (1.0 + math.exp(-z))
w1, w2, b = 0.0, 0.0, 0.0
lr = 0.1
for epoch in range(500):
grad_w1 = grad_w2 = grad_b = 0.0
for x1, x2, y in points:
pred = sigmoid(w1 * x1 + w2 * x2 + b)
error = pred - y # this is dL/d(pre-activation)
grad_w1 += error * x1
grad_w2 += error * x2
grad_b += error
n = len(points)
w1 -= lr * grad_w1 / n
w2 -= lr * grad_w2 / n
b -= lr * grad_b / n
print("learned weights:", w1, w2, b)
for x1, x2, y in points:
pred = sigmoid(w1 * x1 + w2 * x2 + b)
print("true=", y, " predicted_prob=", round(pred, 3))
# --- (b) UNSUPERVISED: the same 10 points, labels never touched below ---
# k-means, k=2: assign each point to its nearest centroid, then move each
# centroid to the mean of the points assigned to it. Repeat.
xy = [(x1, x2) for x1, x2, _ in points]
random.seed(0)
centroids = random.sample(xy, 2) # two random points as starting centroids
for _ in range(10):
clusters = [[], []]
for x1, x2 in xy:
d0 = (x1 - centroids[0][0]) ** 2 + (x2 - centroids[0][1]) ** 2
d1 = (x1 - centroids[1][0]) ** 2 + (x2 - centroids[1][1]) ** 2
clusters[0 if d0 <= d1 else 1].append((x1, x2))
for k in range(2):
if clusters[k]:
mean_x = sum(p[0] for p in clusters[k]) / len(clusters[k])
mean_y = sum(p[1] for p in clusters[k]) / len(clusters[k])
centroids[k] = (mean_x, mean_y)
print("cluster centroids found with no labels at all:", centroids)- Supervised — spam filters, where the label (spam / not spam) comes from user reports, and price or demand forecasting, where the label is the actual realized sale price or units sold. Credit scoring and loan-default prediction follow the same pattern: a lender trains on years of past applications where the true outcome (repaid vs. defaulted) is already on record, then scores new applicants whose outcome hasn't happened yet — the model is only ever as trustworthy as how faithfully those historical outcomes were recorded.
- Unsupervised — customer segmentation, clustering purchase histories into groups with no pre-defined "segment" label, then naming and interpreting the clusters after the fact. Scientific discovery leans on the same trick in the opposite direction: astronomical sky surveys cluster or density-estimate millions of recorded light curves and flag whichever ones sit in an unusually sparse region of that space — candidates for a genuinely new class of astrophysical object that, by definition, no one could have labeled in advance, because no one had ever seen it before.
- Semi-supervised — medical imaging, where a radiologist's confirmed diagnosis is expensive and scarce but raw scans are comparatively cheap to collect; a small labeled set combined with a large unlabeled set can train a substantially better model than the labeled set alone. Bank fraud review works the same way: only the small fraction of transactions that were actually escalated and investigated end up with a confirmed fraud/legitimate tag, while millions of others sit unlabeled, and the cluster assumption — transactions that resemble a confirmed fraud case are more likely fraud themselves — is what lets that unlabeled majority still pull its weight.
- Self-supervised — BERT- and GPT-style language model pretraining (predict a masked or next token, using the surrounding text as its own label) and contrastive image pretraining (predict that two augmented crops came from the same photo), both usually followed by supervised fine-tuning on a much smaller labeled set. Structural biology runs the identical playbook on a different alphabet: pretraining on vast databases of raw, unlabeled protein sequences to learn a general-purpose sense of "what a plausible protein looks like," which is then fine-tuned on the comparatively tiny number of structures that have actually been solved experimentally.
- Reinforcement learning — game-playing agents (reward: win or lose) and ad-serving or recommendation bandits (reward: click or no click), where — unlike a supervised click-through model trained once on a fixed historical log — the bandit's own past choices determine which items it even gets shown feedback on next, forcing it to actively balance exploring uncertain options against exploiting ones it already knows work. Robotics is the physical-world version of the same idea: a warehouse arm learning to grasp objects through repeated trial and error, often in simulation first, off a reward as sparse as "did the object end up in the bin," with no dataset of correct grasping motions ever handed to it up front. Reinforcement learning gets its own dedicated module later in this chapter; it's included here only as a pointer to where it fits relative to the other four.
- "Unsupervised" doesn't mean "assumption-free." Removing labels doesn't remove the need for assumptions — it just moves them into the loss and hypothesis class instead. k-means silently assumes clusters are round and similarly sized; a density model assumes a particular family of distributions. No labels is not the same thing as no bias.
- Semi-supervised isn't "just add more data." Which
x's happen to be labeled is rarely a random sample of the full population — labels tend to exist for the cases that were easy, cheap, or already flagged for review. Treating the unlabeled bulk as freely informative without checking for that label-selection bias can make the combined model worse than the labeled-only baseline, not better. - Self-supervised is not unsupervised. It's easy to lump the two together because neither uses a human-provided label, but self-supervised learning still minimizes a completely ordinary
L(f(x), y)supervised-style loss — the only thing that changed is who wrote the function that generatesy. Confusing the two obscures exactly why self-supervised pretraining transfers so well: it's solving genuinely supervised-style prediction problems, just enormous numbers of cheap, auto-generated ones.
Going deeper
The cleanest modern framing collapses the supervised-versus-unsupervised distinction into a question about where y comes from, not what kind of learning is happening. Large-scale self-supervised pretraining followed by supervised fine-tuning — the recipe behind essentially every current foundation model — spends the overwhelming majority of its compute on an "unsupervised-looking" pile of raw text or images, but every single gradient step inside that pretraining is still ordinary supervised empirical risk minimization against a mechanically generated label. The classical supervised/unsupervised line was never really about the mechanics of optimization; it was about label cost, and in practice that line has mostly dissolved.
Section 2.1.2 (Statistical Decision Theory) makes this precise from the other direction — it shows that the loss function L is not an arbitrary design choice at all, but is forced by the risk you actually want to minimize once you state your problem in decision-theoretic terms. Once that's internalized, the right way to read this entire page is that the five "paradigms" only ever change what data is available to estimate a risk — never the minimization principle in the formula above. Reinforcement learning is the sole exception that additionally changes the object being minimized, a trajectory return instead of a pointwise loss; the other four are, underneath, small variations on exactly the same equation.
A model is trained to predict the missing word in a sentence, and no human ever supplied a label. Is this supervised, unsupervised, or self-supervised learning — and why does the answer matter?
Self-supervised. It's tempting to call it unsupervised because no human-provided label was involved, but the training loop is minimizing a completely ordinary cross-entropy loss between a prediction and a true label — the label just happens to be the word that was mechanically blanked out of the input, rather than something a human annotator wrote down. Genuinely unsupervised methods use losses that don't require a y at all, such as reconstruction error, a clustering distortion, or a density's log-likelihood. The distinction matters because it explains why self-supervised objectives behave like supervised ones in practice, including needing enough of the right kind of data, and being just as capable of overfitting to spurious patterns in that manufactured label as any hand-labeled supervised problem.
Every one of these five paradigms is the same empirical-risk-minimization principle from the formula above, wearing a different data-availability constraint: supervised gets direct (x, y) pairs, unsupervised gets a loss that never needs y, semi-supervised mixes both over the same sample, self-supervised manufactures y from x itself, and reinforcement learning replaces the whole per-example framing with a reward accumulated over a sequence of decisions. Keep that unifying view in mind heading into Section 2.1.2, where the loss function L stops being a design choice and turns out to be forced by decision theory instead.