Bayesian K-means Clustering
Also known as: Bayesian K-means, probabilistic K-means, Dirichlet K-means, BKM
Bayesian K-means clustering extends the classical K-means algorithm by placing prior distributions over cluster centroids and mixing proportions. This probabilistic framework provides uncertainty estimates for cluster assignments, allows principled model selection for the number of clusters, and regularises centroid estimation — especially valuable when data are scarce or high-dimensional.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian K-means when you need uncertainty estimates for cluster memberships, when the number of clusters is unknown and you want a principled selection criterion rather than ad-hoc rules, or when small cluster sizes would make classical centroids unreliable without regularisation. It is particularly appropriate in high-dimensional settings where a prior shrinks noisy centroid estimates. Do not use it when interpretability of hard cluster assignments is paramount and the added modelling complexity is not justified, or when very large datasets make iterative Bayesian updates computationally prohibitive compared to standard K-means or mini-batch alternatives.
Strengths & limitations
- Produces soft, probabilistic cluster assignments rather than hard labels, enabling uncertainty quantification.
- Prior regularisation stabilises centroid estimates for small or imbalanced clusters, reducing sensitivity to outliers.
- Principled model comparison via marginal likelihood allows data-driven selection of the number of clusters K.
- The Dirichlet process extension removes the need to fix K in advance, letting the model infer cluster count from the data.
- Naturally accommodates missing data and heterogeneous cluster sizes within the probabilistic framework.
- Computationally more demanding than classical K-means; variational or MCMC inference adds substantial overhead on large datasets.
- Choice of prior (m0, S0) influences results, particularly when clusters are small or data are sparse — prior sensitivity analysis is needed.
- Assumes Gaussian cluster shapes; non-spherical or non-Gaussian clusters are poorly captured without model extensions.
- Results can be difficult to communicate to non-statistical audiences compared to the intuitive hard-assignment output of standard K-means.
Frequently asked
How is Bayesian K-means different from a Gaussian mixture model?
Gaussian mixture models (GMM) also assign soft memberships and estimate cluster variances, making them the direct probabilistic generalisation of K-means. Bayesian K-means specifically adds prior distributions over the centroids (and mixing weights), enabling regularisation and model comparison. Standard GMM estimation via maximum likelihood (EM) does not include a prior. Full Bayesian GMM estimation and Bayesian K-means are closely related; Bayesian K-means sometimes refers to the special case where cluster covariances are fixed or equal.
What is DP-means and how does it relate to Bayesian K-means?
DP-means (Dirichlet Process K-means) is a hard-assignment clustering algorithm derived by Kulis and Jordan (2012) as the small-variance limit of a Dirichlet process Gaussian mixture model. It automatically determines the number of clusters by adding a new cluster whenever a point is more than a threshold distance from all existing centroids. It retains the simplicity of K-means while inheriting a Bayesian nonparametric justification.
How do I choose the prior in Bayesian K-means?
A common approach is to centre the prior mean m0 on the global data mean and set the prior covariance S0 to a fraction of the data covariance. This weakly informative prior regularises without strongly biasing centroids. Sensitivity analysis — re-running with broader or narrower priors — checks whether conclusions change meaningfully.
When should I prefer standard K-means over its Bayesian version?
Standard K-means is preferable when interpretability of hard assignments is essential, when the dataset is very large and computational cost matters, or when the number of clusters is known with certainty and the Gaussian model assumptions are clearly met. The Bayesian version adds value primarily through uncertainty quantification and regularisation.
Can Bayesian K-means handle non-Gaussian clusters?
The basic model assumes Gaussian clusters. For non-Gaussian or non-convex cluster shapes, extensions such as kernel Bayesian K-means or Bayesian mixture models with non-Gaussian components are needed. If the true clusters are highly non-spherical, the results of standard Bayesian K-means can be misleading regardless of the prior.
Sources
- Kulis, B. & Jordan, M. I. (2012). Revisiting k-means: New algorithms via Bayesian nonparametrics. In Proceedings of the 29th International Conference on Machine Learning (ICML), Edinburgh, Scotland, pp. 513–520. link ↗
- Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer. Chapter 9 (Mixture models and EM) and Chapter 10 (Approximate Inference). ISBN: 978-0387310732
How to cite this page
ScholarGate. (2026, June 3). Bayesian K-means Clustering. ScholarGate. https://scholargate.app/en/statistics/bayesian-k-means-clustering
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Cluster AnalysisStatistics↔ compare
- Bayesian Hierarchical ClusteringStatistics↔ compare
- Bayesian Mixture ModelingStatistics↔ compare
- Cluster AnalysisStatistics↔ compare
- Latent Class AnalysisStatistics↔ compare
- Mixture ModelingStatistics↔ compare