Quadratic Discriminant Analysis (QDA)
Also known as: QDA, quadratic classifier, kuadratik diskriminant analizi
Quadratic discriminant analysis is a generative classifier that models each class with its own multivariate Gaussian distribution, allowing each class a separate covariance matrix. Unlike linear discriminant analysis, which assumes a shared covariance and yields linear boundaries, QDA's per-class covariances produce curved (quadratic) decision boundaries, letting it capture differences in the spread and orientation of the classes.
Key highlights
- Captures classes with different covariances, producing flexible quadratic boundaries.
- Generative model yielding interpretable class posterior probabilities.
- Often outperforms LDA when the equal-covariance assumption is clearly violated.
- Closed-form maximum-likelihood estimation with no iterative tuning.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Choose QDA over LDA when there is evidence that the classes have genuinely different covariance structures — different spreads or orientations — and you have enough training data per class to estimate a full covariance matrix reliably. It works well for moderate numbers of features with ample, roughly Gaussian data and gives calibrated posterior probabilities. With many features or few samples per class the per-class covariance estimates become unstable and QDA overfits; LDA, regularized/shrinkage discriminant analysis, or naive Bayes (which assumes diagonal covariance) are then safer. As a generative Gaussian method it is also sensitive to strong non-normality and outliers, so it pairs well with checks on the class distributions.
Strengths & limitations
- Captures classes with different covariances, producing flexible quadratic boundaries.
- Generative model yielding interpretable class posterior probabilities.
- Often outperforms LDA when the equal-covariance assumption is clearly violated.
- Closed-form maximum-likelihood estimation with no iterative tuning.
- Estimates a full covariance per class, requiring substantially more data than LDA.
- Prone to overfitting when features are many or samples per class are few.
- Assumes class-conditional normality and is sensitive to outliers.
- More parameters reduce interpretability compared with the linear boundaries of LDA.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does QDA differ from LDA?
Both model each class as a Gaussian, but LDA assumes all classes share one covariance matrix, giving linear boundaries, while QDA lets each class have its own covariance, giving quadratic boundaries. QDA is more flexible but estimates many more parameters, so it needs more data and can overfit where LDA would generalize better.
When is QDA likely to overfit?
When the number of features is large relative to the samples per class, because each class's covariance has on the order of p² parameters to estimate. Unstable or near-singular covariance estimates lead to overfitting; regularized discriminant analysis, which shrinks toward a shared covariance, is a remedy.
Does QDA require normally distributed features?
It assumes class-conditional multivariate normality. It is reasonably robust to mild departures, but strong non-normality or outliers degrade its estimates and can distort the quadratic boundaries. Checking the class distributions, transforming features, or using a different classifier helps when normality is badly violated.
Sources
- 1.Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.ISBN 978-0-387-84857-0
- 2.James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An Introduction to Statistical Learning. Springer.ISBN 978-1-4614-7138-7
You have read it. What now?
Cite this page
ScholarGate. (2026, June 2). Quadratic Discriminant Analysis. ScholarGate. https://scholargate.app/machine-learning/quadratic-discriminant-analysis