Linear Discriminant Analysis (LDA)
Linear Discriminant Analysis (Fisher's LDA) · Also known as: LDA, Fisher's discriminant analysis, Fisher linear discriminant, normal discriminant analysis, canonical discriminant analysis
Linear Discriminant Analysis is a supervised method for dimensionality reduction and classification, introduced by Ronald A. Fisher in 1936, that finds linear combinations of features which maximally separate predefined classes while preserving as much class-discriminatory information as possible. It simultaneously serves as a feature-projection technique and a probabilistic classifier, making it one of the foundational methods in pattern recognition and statistical learning.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
LDA is appropriate when the goal is multiclass classification or supervised dimensionality reduction and the data plausibly satisfy its two core assumptions: (1) each class follows a multivariate Gaussian distribution and (2) all classes share the same covariance matrix. It performs well when the number of observations is moderate to large relative to the number of features, and it is especially useful as a preprocessing step before other classifiers when interpretable low-dimensional projections are needed. It is not appropriate when class boundaries are nonlinear, when features are clearly non-Gaussian (e.g., counts or binary indicators without transformation), when classes have very different covariance structures (use QDA instead), or when p is close to or exceeds n (use regularized LDA or PCA first).
Strengths & limitations
- Closed-form solution: no iterative optimization or hyperparameter tuning is required.
- Simultaneous dimensionality reduction and classification in a single coherent framework.
- Computationally efficient and scales well to moderate-dimensional problems.
- The discriminant axes are interpretable linear combinations of the original features.
- Naturally handles multiclass problems, projecting to at most K-1 dimensions.
- Provides class posterior probabilities under the Gaussian assumption, enabling calibrated uncertainty estimates.
- Assumes multivariate Gaussian class-conditional distributions, which many real datasets violate.
- The equal-covariance assumption is often unrealistic; severe violations inflate error rates.
- Sensitive to outliers because sample means and covariances are not robust estimators.
- Breaks down when p is close to or larger than n because S_W becomes singular or ill-conditioned.
- Linear decision boundaries cannot capture curved or complex class separations.
- Prior class probabilities must be specified or estimated from possibly unrepresentative training data.
Frequently asked
What is the difference between LDA and PCA?
PCA is unsupervised and finds the directions of maximum total variance regardless of class labels. LDA is supervised and finds the directions that maximize class separation relative to within-class spread. PCA preserves the most information about the data; LDA preserves the most information about the class structure. For classification tasks, LDA projections are typically more discriminative than PCA projections.
When should I use QDA instead of LDA?
Quadratic Discriminant Analysis (QDA) relaxes the equal-covariance assumption and fits a separate covariance matrix per class, producing curved (quadratic) decision boundaries. Use QDA when classes clearly differ in their spread or shape and you have enough observations per class to estimate each covariance matrix reliably. LDA is preferred when the equal-covariance assumption is reasonable or when sample sizes are small and the extra parameters of QDA would lead to overfitting.
How many discriminant dimensions should I retain?
At most K-1 discriminant axes carry class-discriminatory information for K classes, so for binary classification there is exactly one discriminant axis. In practice, plot the eigenvalues in descending order and retain the smallest number of axes that together explain a high proportion (e.g., 95%) of the between-class variance. The first one or two axes are often sufficient for visualization and classification.
What should I do if p is large relative to n?
When the number of features p is close to or exceeds the number of observations n, the within-class scatter matrix S_W becomes singular and cannot be inverted. Standard remedies are: (1) apply PCA first to reduce dimensionality, then run LDA on the principal components; (2) use regularized LDA (e.g., Friedman's 1989 regularized discriminant analysis), which shrinks S_W toward a scaled identity matrix; or (3) use a penalized variant such as sparse LDA.
Sources
- Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2), 179–188. DOI: 10.1111/j.1469-1809.1936.tb02137.x ↗
- Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd ed., Ch. 4). Springer. ISBN: 978-0-387-84857-0
How to cite this page
ScholarGate. (2026, June 3). Linear Discriminant Analysis (Fisher's LDA). ScholarGate. https://scholargate.app/en/machine-learning/linear-discriminant-analysis
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Logistic RegressionResearch Statistics↔ compare
- Naive BayesMachine learning↔ compare
- Quadratic Discriminant AnalysisMachine learning↔ compare
- Random ForestMachine learning↔ compare