Bayesian Discriminant Analysis
Also known as: BDA, Bayesian linear discriminant analysis, Bayesian quadratic discriminant analysis, Bayesian classification
Bayesian discriminant analysis assigns observations to predefined groups by combining a multivariate Gaussian likelihood for each class with prior distributions over the class means and covariance matrices. Posterior predictive probabilities replace point-estimate decision boundaries, providing principled uncertainty quantification for classification in small or high-dimensional samples.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian discriminant analysis when you need group membership probabilities with principled uncertainty, when your training sample is small relative to the number of predictors (where classical LDA/QDA estimates become unstable), or when you have informative prior knowledge about class distributions that you want to incorporate formally. It is also preferable when calibrated probability outputs — not just labels — matter for downstream decisions. Do not use it when class boundaries are non-linear and non-Gaussian (prefer kernel methods or tree-based classifiers), when computational overhead of integrating over parameters is prohibitive and sample sizes are large (classical LDA is accurate and far faster), or when the multivariate normality assumption is severely violated.
Strengths & limitations
- Posterior predictive probabilities are inherently calibrated and quantify classification uncertainty rather than providing only a hard label.
- Conjugate Normal-Inverse-Wishart prior yields a closed-form posterior, making the method analytically tractable and computationally efficient even without MCMC.
- Prior distributions act as regularizers, stabilising estimates when sample size is small or predictors are numerous relative to observations.
- Principled framework for incorporating domain knowledge through informative priors on class means and covariances.
- Naturally handles the small-n, large-p regime better than maximum-likelihood LDA, which can be ill-conditioned or degenerate.
- The multivariate normality assumption within each class may not hold, especially for skewed, bounded, or categorical predictors.
- Prior specification requires care: a weakly informative or misspecified prior can dominate small-sample posteriors and bias classification.
- The homoscedasticity assumption of the linear variant (shared covariance across classes) may be violated, requiring the quadratic variant and more parameters.
- Scales less easily than modern machine-learning classifiers to very high-dimensional feature spaces without additional sparsity priors.
- Interpretation of the posterior predictive and prior hyperparameters demands statistical sophistication that may not be available in applied teams.
Frequently asked
How does Bayesian discriminant analysis differ from classical LDA?
Classical LDA estimates class means and the pooled covariance by maximum likelihood and then draws a linear boundary. Bayesian discriminant analysis places a prior over those parameters and classifies via the posterior predictive probability, which averages over parameter uncertainty. The result is better-calibrated probabilities and more stable boundaries when sample sizes are small.
Do I need MCMC to implement Bayesian discriminant analysis?
Not in the standard Normal-Inverse-Wishart conjugate setting. The conjugate prior yields a closed-form posterior and a closed-form posterior predictive (a multivariate t distribution for each class), so MCMC is unnecessary. MCMC is needed only when non-conjugate priors or non-Gaussian likelihoods are used.
How do I choose the prior hyperparameters?
A common default is a weakly informative prior with small prior sample sizes (kappa and nu just above the minimum for a proper posterior) and a prior mean set to the overall sample mean. When genuine domain knowledge exists — for example, published class means from prior studies — it can be encoded in the hyperparameters. Sensitivity analysis across a range of hyperparameter values is always advisable.
What is the difference between the linear and quadratic Bayesian variants?
The linear variant (Bayesian LDA) assumes all classes share the same covariance matrix, which is estimated from pooled data. The quadratic variant (Bayesian QDA) allows each class its own covariance. QDA requires estimating more parameters and needs larger per-class samples; in the Bayesian setting the class-specific prior covariances offer regularization, but QDA is still preferable only when the homoscedasticity assumption is clearly violated.
Can Bayesian discriminant analysis handle more than two classes?
Yes. The posterior predictive approach extends naturally to K > 2 classes: a separate Normal-Inverse-Wishart posterior is computed for each class, and the predicted class is the one with the highest posterior predictive density times the prior class probability. No binary decomposition is needed.
Sources
How to cite this page
ScholarGate. (2026, June 3). Bayesian Discriminant Analysis. ScholarGate. https://scholargate.app/en/statistics/bayesian-discriminant-analysis
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Cluster AnalysisStatistics↔ compare
- Bayesian Confirmatory Factor AnalysisPsychometrics↔ compare
- Bayesian Latent Class AnalysisStatistics↔ compare
- Discriminant AnalysisStatistics↔ compare