Linear Discriminant Analysis (LDA — Classification)
Also known as: LDA, Fisher's LDA, Fisher's linear discriminant, discriminant function analysis, Doğrusal Diskriminant Analizi (LDA — Sınıflandırma)
Linear Discriminant Analysis (LDA) is a parametric supervised classification method that finds the linear combination of continuous predictors that best separates two or more predefined groups. Introduced by Ronald A. Fisher in his landmark 1936 paper on taxonomic measurements, it simultaneously serves as a classifier and a dimensionality-reduction tool, and can be understood as the classification-oriented counterpart of MANOVA.
Key highlights
- Simultaneously classifies and reduces dimensionality, producing interpretable canonical axes that can be visualised in two dimensions.
- Computationally efficient and analytically tractable, with a closed-form solution via the generalised eigenvalue problem.
- Optimal Bayes classifier when the normality and equal-covariance assumptions hold exactly.
- Produces probabilistic class memberships alongside hard classifications, enabling nuanced uncertainty reporting.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use LDA when you need to classify observations into two or more predefined groups using a set of continuous predictors, or when you want to reduce dimensionality before further analysis. Four assumptions must hold: (1) multivariate normality of the predictors within each class — check with Mardia's or Henze-Zirkler's test; (2) homogeneity of within-class covariance matrices — check with Box's M test; (3) independence of observations; and (4) no severe multicollinearity among predictors, which destabilises the matrix inversion. A minimum of approximately 50 observations is recommended. When the equal-covariance assumption fails, Quadratic Discriminant Analysis (QDA) is the natural alternative; for binary outcomes with relaxed distributional assumptions, logistic regression is a robust option.
Strengths & limitations
- Simultaneously classifies and reduces dimensionality, producing interpretable canonical axes that can be visualised in two dimensions.
- Computationally efficient and analytically tractable, with a closed-form solution via the generalised eigenvalue problem.
- Optimal Bayes classifier when the normality and equal-covariance assumptions hold exactly.
- Produces probabilistic class memberships alongside hard classifications, enabling nuanced uncertainty reporting.
- Sensitive to violations of multivariate normality and equal covariance; QDA or nonparametric methods should be preferred when these fail.
- Linear decision boundaries may be inadequate for groups with complex, nonlinear separation in the predictor space.
- High multicollinearity among predictors makes the within-class covariance matrix ill-conditioned and the discriminant weights unstable.
- Requires at least 50 observations and performs poorly when the number of predictors approaches or exceeds the number of observations.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does LDA differ from logistic regression?
Both produce linear classifiers, but LDA models the joint distribution of predictors within each class (generative approach) while logistic regression models the conditional probability of class membership directly (discriminative approach). LDA is more efficient when the normality and equal-covariance assumptions hold; logistic regression is more robust when they do not.
What is the difference between LDA for classification and LDA for dimensionality reduction?
They use the same mathematical machinery. In classification mode, LDA assigns observations to groups. In dimensionality reduction mode, LDA projects high-dimensional data onto the discriminant axes (canonical variates) for visualisation or as a preprocessing step before another classifier. Many implementations do both simultaneously.
When should I use QDA instead of LDA?
Use Quadratic Discriminant Analysis when Box's M test indicates that the within-class covariance matrices differ significantly across groups. QDA fits a separate covariance matrix per class, yielding quadratic (curved) decision boundaries. It requires larger sample sizes to estimate those additional parameters reliably.
How do I evaluate classification performance?
Report the full confusion matrix alongside overall accuracy. Use k-fold cross-validation (typically k = 10) to estimate generalisation accuracy on unseen data, as training-set accuracy is optimistically biased. For imbalanced classes, report sensitivity, specificity, and the area under the ROC curve per class rather than relying on overall accuracy alone.
Sources
- 1.Fisher, R.A. (1936). The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7(2), 179–188.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 1). Linear Discriminant Analysis (Classification). ScholarGate. https://scholargate.app/statistics/lda-classification