Generative vs. Discriminative Models
Modeling P(x,y) vs. P(y|x) directly, and when each one actually wins.
On this page
Beginner: every classifier in this module answers "what's the label?" but they get there by two fundamentally different routes. Logistic and softmax regression (Module 4) go straight for the answer: given the features, what's the probability of each class? Naive Bayes and Gaussian Discriminant Analysis (sections 2.5.1 and 2.5.2) take a detour: for each possible label, they first ask "if this really were the answer, what would the features look like?" — building a little generative story for each class — and only then use Bayes' rule to flip that story around into a prediction. The first family is called discriminative; the second is called generative, because it can literally generate data, not just classify it. This lesson is where those two philosophies finally meet head-to-head.
Intermediate: the detour a generative model takes isn't wasted effort — it buys real capabilities a discriminative model structurally cannot have, because a discriminative model never builds a model of x at all. A generative model can draw brand-new synthetic examples from its learned P(x|y), it can compute P(x) itself to flag data that looks unlike anything it was trained on, and it can answer questions about a partially-observed input by marginalizing out whatever wasn't measured. None of that is available to a model that only ever learned P(y|x). In exchange, the generative model is betting real accuracy on its assumptions about x being roughly right — and that bet is the crux of the trade-off this lesson formalizes.
Advanced: stated most precisely, a generative classifier models the joint distribution P(x, y) — equivalently, the pair {P(x|y), P(y)} — and derives P(y|x) at prediction time via Bayes' rule; a discriminative classifier parametrizes and fits P(y|x) directly, with no distributional assumption on x anywhere in the model. Section 2.5.2 already showed one exact instance of the relationship between them: LDA's posterior, once its Gaussian assumptions are plugged into Bayes' rule, comes out algebraically identical in form to logistic regression's sigmoid-of-a-linear-function posterior — meaning LDA's entire hypothesis class sits strictly inside logistic regression's. What LDA gains by committing to more structure than that superset requires is the subject of Ng & Jordan's classic asymptotic result, worked through rigorously below: fewer assumptions eventually win as data accumulates, but more assumptions — when even roughly right — win first.
Generative — model the joint, then invert it with Bayes' rule:
Discriminative — model the conditional directly, and stop there:
Both routes can produce the exact same decision rule at prediction time — pick the y maximizing P(y|x) — but only the left-hand side ever builds a model of x itself, and everything in this lesson follows from that single asymmetry.
Both families ultimately want the same object at prediction time — P(y|x), or just the label that maximizes it. The difference is entirely in how that object is obtained. A discriminative model writes down a parametric form for P(y|x) directly — h_\theta(x) in logistic regression, a softmax in 2.4.2 — and fits θ by maximizing the conditional likelihood Πᵢ P(yᵢ|xᵢ; θ). The marginal distribution of the inputs, P(x), never appears anywhere in that objective; it is treated as fixed and irrelevant, exactly the same way x is treated as "given" (not modeled) in ordinary least squares (section 2.3.1).
A generative model instead writes down the joint distribution by factoring it as P(x,y) = P(x|y)P(y), fits the class-conditional density P(x|y=k) and the class prior P(y=k) separately — almost always by MLE, exactly as section 2.5.1's Naive Bayes and section 2.5.2's GDA both do — and only then recovers the posterior at prediction time via Bayes' rule:
The denominator here, P(x) = \sum_j P(x|y=j)P(y=j), is exactly the marginal a discriminative model refuses to touch — and having it in hand is what unlocks three capabilities a discriminative model has no path to at all, because none of them are expressible without some model of x.
1. Sampling new data. Since P(x|y=k) is an actual, fully specified probability distribution (a product of Gaussians and Bernoullis/Multinomials for Naive Bayes, a multivariate Gaussian for GDA), it can be sampled from directly: draw a class k ~ P(y), then draw x ~ P(x|y=k). For a fitted GDA model this is concrete and mechanical — draw a standard normal vector z, transform it by the fitted covariance's Cholesky factor L (so LL^\top = \hat\Sigma), and shift by the fitted class mean: x = \hat\mu_k + Lz. The first diagram below does exactly this, live, from a GDA model fit on this lesson's own synthetic data — every point it draws is a genuinely new (x₁, x₂) pair that never appeared in training. A discriminative model has no analogous procedure: h_\theta(x) is a function from x, not a distribution over x, so there is nothing to sample.
2. Computing P(x) itself, for outlier / novelty detection. The denominator above, P(x) = \sum_j P(x|y=j)P(y=j), is a byproduct a generative model gets for free — it is exactly the normalizing constant Bayes' rule needs. A point x that lands in a region of very low P(x) under every class is flagged as unlike anything the model has seen, independent of what label gets predicted for it — this is the standard generative approach to anomaly and novelty detection (a preview of the outlier-detection material later in the syllabus). A discriminative model has no such quantity anywhere in its machinery: it can report high confidence in a label for an input that is wildly outside its training distribution, with nothing internal to flag that the input itself looked strange.
3. Handling missing features by marginalization. Suppose x is split into an observed part x_{obs} and a missing part x_{mis}. A generative model can still compute a valid posterior over y by integrating the missing coordinate out of the joint density:
For a multivariate Gaussian class-conditional this integral has a clean closed form used throughout statistics: the marginal of a Gaussian over any subset of its coordinates is again Gaussian, with the mean and covariance simply restricted to the observed coordinates — no numerical integration required. Concretely, if x = (x₁, x₂) and x₂ is missing, P(x_1 \mid y=k) is just \mathcal{N}(\mu_{k,1}, \Sigma_{11}) — the first coordinate of the class mean and the top-left entry of the covariance matrix, both already estimated during fitting. The third diagram below makes this concrete with a slider that removes a query point's second coordinate and shows GDA's posterior update correctly via exactly this projected 1-D Gaussian. A discriminative model has no marginalization rule available at all, because it never had a joint density to marginalize in the first place — the only options are to refuse to answer or to impute a guess for the missing value and hope the guess doesn't bias the prediction.
Where this is used: all three of these are consequences of one structural fact — a generative model's parameters define a full probability distribution over (x, y), while a discriminative model's parameters define only a function of x. Everything else in this lesson is about what that structural difference costs and buys in terms of prediction accuracy itself.
Fix the same binary classification setup section 2.5.2 used: two classes, each x|y=k ~ N(μₖ, Σ) sharing one covariance matrix, with class priors P(y=1)=φ, P(y=0)=1-φ. Two very different amounts of machinery are available to fit this same problem.
What LDA estimates. In d dimensions: two mean vectors μ₀, μ₁ (2d numbers), one shared covariance matrix Σ (d(d+1)/2 free numbers, by symmetry), and one scalar prior φ — a total of 2d + d(d+1)/2 + 1 parameters, every one of them fit by the closed-form MLEs derived in section 2.5.2 (empirical class means, pooled empirical covariance, empirical class fraction). Critically, this parametrization assumes the Gaussian family and the shared-covariance structure — it is not free to represent an arbitrary class-conditional shape.
What logistic regression estimates. Just θ ∈ ℝ^{d+1} — d+1 numbers, fit by the gradient-descent or Newton iteration of section 2.4.1, with no distributional assumption on x anywhere in the model. It never asks whether the features look Gaussian, correlated, skewed, or anything else — it only ever fits the shape of the boundary between classes.
The exact connection between them. Section 2.5.2 already derived that plugging the shared-covariance Gaussian class-conditionals into Bayes' rule produces a posterior of the form:
with w = \Sigma^{-1}(\mu_1-\mu_0) and w₀ collecting the remaining constant and prior terms — exactly the sigmoid-of-a-linear-function form logistic regression assumes directly, as a modeling choice, from the start. That means every decision boundary LDA can ever produce is one particular member of the family logistic regression can represent — LDA's hypothesis class is a strict subset of logistic regression's. The extra structure LDA carries — the full Gaussian generative story, not just the boundary it implies — is exactly the d(d+1)/2 covariance parameters and the 2d mean parameters that get thrown away once you only look at the induced w and w₀.
This is the geometric heart of the whole comparison: LDA is logistic regression plus a strong, specific claim about how the two classes' features are distributed. When that claim is true, LDA is estimating the same boundary with a more efficient, lower-variance route — it never has to search a whole linear hypothesis class by iterative optimization, it reads the boundary straight off two means and a shared covariance. When that claim is false, LDA is still forced to report a linear boundary of the same restricted form, while logistic regression's weaker assumptions let it fit whatever linear boundary the conditional likelihood actually prefers — unconstrained by any story about the shape of P(x|y).
Where this is used: "more assumptions but a subset hypothesis class" is precisely the setup the bias-variance trade-off (section 2.1.3) describes in the abstract: a restricted hypothesis class has less capacity to overfit (lower variance) but pays for it with bias whenever the restriction doesn't hold. The next Derivation makes that trade-off numerically precise as a function of sample size.
Ng and Jordan's 2001 result ("On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes") makes the LDA-vs-logistic-regression story above precise, and generalizes it to the naive Bayes / logistic regression pair covered in section 2.5.1. Stating it carefully requires being explicit about a word — "asymptotic" — that is doing two different jobs in the same sentence.
Meaning 1: as the number of training examples n → ∞. This is the familiar sense — what happens to a fitted model's test error as it sees more and more data.
Meaning 2: the fixed error level each model's assumptions converge TO. Every model — generative or discriminative — has its own asymptotic error floor: the error it would achieve with infinite data, given the restrictions baked into its hypothesis class. For a correctly-specified discriminative model (logistic regression, when the true P(y|x) really is a sigmoid of a linear function) that floor is exactly the Bayes error — the true, irreducible error of the actual data-generating process. For a generative model whose distributional assumption is wrong (e.g. naive Bayes when features are not actually conditionally independent, or GDA when the true class-conditionals are not actually Gaussian, or not actually sharing one covariance), that floor can be strictly worse than the Bayes error — infinite data cannot fix a wrong assumption about the shape of P(x|y), it can only reveal exactly how wrong that assumption's consequences are.
The core finding. With that distinction in hand, the result has two parts:
(a) As n → ∞ (meaning 1), the discriminative model's error converges to its own asymptotic floor (meaning 2) — which, whenever its hypothesis class contains the true P(y|x), equals the Bayes-optimal error itself, since it was never handicapped by any assumption on P(x) that could be wrong. The generative model converges to its own floor too — but that floor can be strictly higher whenever its distributional assumption about P(x|y) is even slightly misspecified. So for large enough n, the discriminative model's error is never worse, and is often strictly better.
(b) At small n, the ordering usually flips. Ng & Jordan show — for the naive Bayes / logistic-regression pair, using the fact that they share the same parametric form of decision boundary once naive Bayes's exponential-family class-conditionals are plugged into Bayes' rule (an argument that runs exactly parallel to the LDA case above) — that the generative model's test error approaches its own asymptotic floor at a rate of O(log d) training examples, where d is the number of features, while the discriminative model needs O(d) examples to approach its own floor. Since log d ≪ d for any reasonable feature count, the generative model's error typically drops to somewhere near its (possibly worse) limit almost immediately, while the discriminative model's error is still falling well after the generative model's curve has gone flat — producing exactly the crossover the learning-curve diagram below computes: the generative model wins early, and the discriminative model catches up or overtakes it later.
The intuition, in bias-variance terms (section 2.1.3). A generative model's distributional assumption acts as a strong, hard-coded prior about the shape of the data — exactly the kind of inductive bias that reduces variance at the cost of some bias. With very few examples, variance dominates test error (there simply isn't enough data yet to pin down a large, loosely-constrained hypothesis class like logistic regression's), so the lower-variance generative model wins even though its bias is nonzero. As n grows, variance shrinks toward zero for both models regardless of hypothesis-class size, and the comparison is decided almost entirely by which model's bias is smaller — which favors the discriminative model whenever the generative assumption isn't exactly right, and ties when it is.
Where this is used: this is the single most practically important takeaway of the whole module — not "generative models are old-fashioned" or "discriminative models are strictly better," but a genuine, quantifiable trade governed by how much data is available and how much the modeler trusts the generative assumptions. The learning-curve diagram below computes this crossover directly rather than merely asserting it.
To make Ng & Jordan's result something you can see rather than only cite, fix a known true distribution: two classes, each Gaussian in two dimensions, x \mid y=0 \sim \mathcal{N}(\mu_0, \Sigma), x \mid y=1 \sim \mathcal{N}(\mu_1, \Sigma), with a shared covariance carrying real correlation between the two features (this is precisely the setup used by every diagram and code example in this lesson). Because the covariance really is shared and the class-conditionals really are Gaussian, GDA's assumptions are exactly correct for this data — the favorable case for the generative model, chosen deliberately so the crossover the theory predicts shows up cleanly rather than being swamped by misspecification.
For a sequence of training-set sizes n ranging from 4 up to 512, the learning- curve diagram below repeatedly: draws a fresh seeded training sample of size n from this known distribution, fits GDA by the closed-form MLEs of section 2.5.2 (empirical means, pooled covariance, empirical prior) and logistic regression by the gradient descent of section 2.4.1, and scores both on one large, fixed, seeded held-out test set drawn from the same distribution — averaging over several repeated draws at each n to smooth out sampling noise. Because both classes share one covariance and are genuinely Gaussian, both models' decision boundaries converge to the same line as n → ∞ — the shared-covariance case is exactly where GDA's asymptotic floor equals logistic regression's, so this example isolates the small-sample effect cleanly, without also mixing in an asymptotic-floor gap.
What the computed curves show is the qualitative Ng & Jordan pattern directly: at the smallest sample sizes, GDA's test error is noticeably lower — with only a handful of points per class, the empirical class means and pooled covariance are still reasonably stable estimates of two means and a shared covariance matrix, while logistic regression's gradient descent has barely enough signal to pin down even a 3-parameter linear boundary reliably. As n grows into the dozens and then hundreds, logistic regression's error keeps falling and the two curves converge — visibly crossing or merging well before n reaches a few hundred, exactly the O(log d) vs. O(d) convergence- rate gap predicted above, made concrete for d = 2.
Left: GDA's fitted class-conditional Gaussians, with brand-new synthetic points being drawn live from those densities via a seeded PRNG. Right: the same problem's discriminative decision boundary -- classifies just as well, but has no density anywhere to sample from.
Both curves are real: GDA and logistic regression are actually fit on repeated seeded samples of increasing size n and scored on one fixed held-out set. GDA wins at tiny n; the curves converge as n grows, since the shared-covariance assumption is exactly true for this synthetic data.
Toggle the query point's second coordinate to 'missing' and watch GDA fall back to the 1-D marginal densities P(x1|y), still producing a valid posterior by integrating x2 out of the fitted joint Gaussian analytically. The discriminative boundary has no such option.
All three tabs fit the same head-to-head comparison: a generative model against logistic regression, on the same known Gaussian source, at the same growing sequence of sample sizes — the exact computation the learning-curve diagram above renders. Expect the generative model's error to lead at the smallest n and the gap to close as n grows.
#include <cmath>
#include <iostream>
#include <random>
#include <vector>
using Vec = std::vector<double>;
using Mat = std::vector<Vec>;
double sigmoid(double z) { return 1.0 / (1.0 + std::exp(-z)); }
struct GDA {
Vec mu0{0, 0}, mu1{0, 0};
Mat covInv{{0, 0}, {0, 0}};
double phi = 0.5;
};
// Fits the shared-covariance Gaussian MLEs (section 2.5.2), then inverts the 2x2 covariance.
GDA fitGDA(const std::vector<Vec>& X, const std::vector<int>& y) {
GDA m;
int n0 = 0, n1 = 0;
for (size_t i = 0; i < X.size(); ++i) {
if (y[i] == 0) { m.mu0[0] += X[i][0]; m.mu0[1] += X[i][1]; n0++; }
else { m.mu1[0] += X[i][0]; m.mu1[1] += X[i][1]; n1++; }
}
m.mu0[0] /= n0; m.mu0[1] /= n0;
m.mu1[0] /= n1; m.mu1[1] /= n1;
m.phi = double(n1) / (n0 + n1);
double s00 = 0, s01 = 0, s11 = 0;
for (size_t i = 0; i < X.size(); ++i) {
const Vec& mu = (y[i] == 0) ? m.mu0 : m.mu1;
double d0 = X[i][0] - mu[0], d1 = X[i][1] - mu[1];
s00 += d0 * d0; s01 += d0 * d1; s11 += d1 * d1;
}
int n = n0 + n1;
s00 /= n; s01 /= n; s11 /= n;
double det = s00 * s11 - s01 * s01;
m.covInv = {{ s11 / det, -s01 / det }, { -s01 / det, s00 / det }};
return m;
}
int gdaPredict(const GDA& m, double x0, double x1) {
auto quad = [&](const Vec& mu) {
double d0 = x0 - mu[0], d1 = x1 - mu[1];
return d0 * (m.covInv[0][0] * d0 + m.covInv[0][1] * d1) +
d1 * (m.covInv[1][0] * d0 + m.covInv[1][1] * d1);
};
double logOdds = 0.5 * quad(m.mu0) - 0.5 * quad(m.mu1) + std::log(m.phi / (1 - m.phi));
return logOdds >= 0 ? 1 : 0;
}
Vec fitLogReg(const std::vector<Vec>& X, const std::vector<int>& y, double lr, int iters) {
// The exact gradient Xᵀ(σ(Xθ)−y) derived in section 2.4.1, bias column included.
Vec theta = {0.0, 0.0, 0.0};
for (int it = 0; it < iters; ++it) {
Vec g = {0.0, 0.0, 0.0};
for (size_t i = 0; i < X.size(); ++i) {
double z = theta[0] + theta[1] * X[i][0] + theta[2] * X[i][1];
double err = sigmoid(z) - y[i];
g[0] += err; g[1] += X[i][0] * err; g[2] += X[i][1] * err;
}
for (int j = 0; j < 3; ++j) theta[j] -= (lr / X.size()) * g[j];
}
return theta;
}
int logregPredict(const Vec& theta, double x0, double x1) {
double z = theta[0] + theta[1] * x0 + theta[2] * x1;
return sigmoid(z) >= 0.5 ? 1 : 0;
}
int main() {
std::mt19937 rng(0);
std::normal_distribution<double> stdNormal(0.0, 1.0);
Vec mu0 = {-1.1, -0.6}, mu1 = {1.3, 0.9};
// Cholesky factor of the shared true covariance [[1.1, 0.55], [0.55, 0.85]].
double L11 = std::sqrt(1.1), L21 = 0.55 / L11, L22 = std::sqrt(0.85 - L21 * L21);
auto makeData = [&](int n) {
std::vector<Vec> X(n, Vec(2));
std::vector<int> y(n);
for (int i = 0; i < n; ++i) {
y[i] = i % 2;
const Vec& mu = y[i] == 0 ? mu0 : mu1;
double z1 = stdNormal(rng), z2 = stdNormal(rng);
X[i][0] = mu[0] + L11 * z1;
X[i][1] = mu[1] + L21 * z1 + L22 * z2;
}
return std::make_pair(X, y);
};
auto [Xtest, ytest] = makeData(4000);
for (int n : {8, 32, 128, 512}) {
auto [Xtrain, ytrain] = makeData(n);
GDA gda = fitGDA(Xtrain, ytrain);
Vec theta = fitLogReg(Xtrain, ytrain, 0.4, 400);
int gdaWrong = 0, logregWrong = 0;
for (size_t i = 0; i < Xtest.size(); ++i) {
if (gdaPredict(gda, Xtest[i][0], Xtest[i][1]) != ytest[i]) gdaWrong++;
if (logregPredict(theta, Xtest[i][0], Xtest[i][1]) != ytest[i]) logregWrong++;
}
std::cout << "n=" << n
<< " GDA err=" << double(gdaWrong) / Xtest.size()
<< " logreg err=" << double(logregWrong) / Xtest.size() << "\n";
}
return 0;
}- Small-sample medical and clinical studies. When a study has dozens or low-hundreds of patients — not the millions of an internet-scale dataset — a generative model's stronger assumptions act as exactly the regularization needed to get a stable, usable classifier; naive Bayes and GDA remain genuinely competitive baselines here for precisely this reason.
- Semi-supervised learning. A generative model can incorporate unlabeled
x's directly, because it modelsP(x): unlabeled examples still inform the estimate of the class-conditional densities' shared structure (e.g. via EM, section 2.7's territory) even without a label attached. A discriminative model has no mechanism to use an unlabeled example at all, since its objective only ever conditions onybeing present. - Anomaly / novelty / fraud detection. Whenever the actual task needs
P(x)itself — flagging a transaction, a sensor reading, or a network request that looks unlike anything seen before, independent of any label — only a generative model has that quantity available at all (a preview of the outlier-detection material later in this syllabus). - Text classification with Naive Bayes. Spam filtering and topic classification on modest labeled datasets remain classic naive-Bayes territory (section 2.5.1) — fast to fit, robust with limited labeled examples, and easy to update online as new documents arrive one at a time.
- Large-scale industrial classification and ranking. Once
nis in the millions — ad click prediction, search ranking, large-scale content moderation — the discriminative model's advantage from the Ng & Jordan analysis has fully kicked in: raw classification accuracy is what gets graded, sample size is no longer the bottleneck, and a weakly-assumption-laden discriminative model (logistic regression, gradient-boosted trees, deep networks) reliably outperforms a generative alternative unless there's a specific non-classification reason (sampling, missing data,P(x)itself) to want one. - Modern deep learning's supervised core. The overwhelming majority of production deep neural classifiers — image classifiers, most language-model fine-tuning for classification/ranking heads — are trained discriminatively: they optimize a conditional (usually cross-entropy) loss and never explicitly model
P(x), precisely because in the large-data regime this analysis predicts that is the higher-accuracy route.
- Concluding "discriminative is just strictly better" from the
n → ∞half of the Ng & Jordan result alone. That half is only true in the large-sample limit; at smalln, the generative model very often wins outright, exactly as the learning-curve diagram above computes rather than merely claims. - Reading a generative model's strong small-sample performance as evidence that its distributional assumptions are TRUE. They aren't being validated by that result — the good small-
nperformance is a variance-reduction effect from a useful inductive bias, not confirmation that features really are conditionally independent (naive Bayes) or really are jointly Gaussian (GDA). A model can be "wrong but useful" at smallnand still be measurably worse than a correctly-specified discriminative alternative oncenis large enough for that bias to dominate. - Forgetting that a generative model's asymptotic error (meaning 2 from the Derivation above — its fixed floor) can be strictly WORSE than a well-specified discriminative model's, if its distributional assumptions are wrong. More data does not rescue a misspecified generative model from this floor; it only reveals the floor more precisely. Whenever there's real doubt about the class-conditional assumptions and
nis not small, that is a real argument for preferring the discriminative model, not just a theoretical footnote. - Treating "generative" and "discriminative" as describing where a model sits on some single quality axis, rather than as a design choice about which quantity gets modeled. k-Nearest Neighbors (section 2.5.3) is neither — it approximates
P(y|x)non-parametrically from the local neighborhood, with no explicit density estimate ofP(x|y)and no single global parametric form forP(y|x)either — a reminder that this is a spectrum of modeling choices, not a strict binary.
Going deeper
Hybrid objectives, and where this tension resurfaces in modern generative modeling. The generative/discriminative split is not always an either/or choice at training time. A hybrid objective can fit the same generative model's parameters while explicitly trading off between maximizing the full joint log-likelihood \sum_i \log P(x_i, y_i) (the fully generative objective) and the conditional log-likelihood \sum_i \log P(y_i \mid x_i) (the fully discriminative one) via a weighted combination of the two — nudging a generative model's fit toward better classification accuracy without discarding the generative structure (and hence without losing sampling, P(x), or missing-data handling). This is a genuine, continuously tunable dial between the two philosophies covered in this lesson as if they were a strict binary.
The same generative/discriminative distinction resurfaces, at much larger scale, in modern deep learning — even though this syllabus's classical-ML scope stops short of it. The overwhelming majority of production deep classifiers are trained purely discriminatively, exactly matching the large-n conclusion above. But an entire separate family of models — variational autoencoders, diffusion models, GANs, and autoregressive language models — is generative in precisely this lesson's sense: each one explicitly represents (or learns to sample from) something close to P(x), or P(x|y) for conditional variants, rather than only a decision boundary. That's not a stylistic choice; it's a structural necessity, because "generate a new image" or "generate the next token of text" are tasks that are only expressible at all in terms of a model of P(x) — a discriminative model, by construction, has nothing to offer them, for exactly the same reason the first diagram in this lesson shows a discriminative boundary alone has nothing to sample from.
A generative model with a distributional assumption you know to be technically wrong (e.g. naive Bayes's conditional-independence assumption, when the features are actually somewhat correlated) can still outperform a correctly-specified discriminative model in practice. Under what condition does that advantage go away?
It goes away as the training-set size n grows large. At small n, variance dominates test error, and the generative model's (wrong but structured) assumption acts as an inductive bias that reduces variance -- exactly the bias-variance trade-off of section 2.1.3 -- so it wins even though its asymptotic error floor (the fixed point its assumptions converge to) may be strictly worse than the Bayes-optimal error the correctly-specified discriminative model eventually reaches. As n increases, variance shrinks toward zero for both models, the comparison is decided almost entirely by bias, and the discriminative model's smaller (here, zero) bias lets it overtake and then permanently beat the generative model once n is large enough -- exactly the crossover the learning-curve diagram in this lesson computes directly. The advantage is real but temporary: it is a small-sample-size phenomenon, not a permanent one, whenever the generative assumption is actually wrong.
The choice between generative and discriminative isn't a question of which family is "better" in the abstract — it's a question of how much training data you actually have, how much you trust your assumptions about P(x|y), and whether you need more from the model than a classification decision (sampling, P(x) for novelty detection, graceful handling of missing features). That's the thread running through this entire module: Naive Bayes and GDA (2.5.1, 2.5.2) show what a strong generative assumption buys, k-NN (2.5.3) shows a classifier that commits to neither philosophy at all, and this lesson makes the trade-off between the two philosophies rigorous rather than a matter of taste.
Module 6, Kernel Methods & Support Vector Machines, returns fully to the discriminative side of this divide — but pushes it in a direction nothing so far has tried: instead of modeling a probability at all, it fits a decision boundary directly by maximizing geometric margin, and uses the kernel trick to do so in feature spaces far too large to ever construct explicitly.