Bayesian Hierarchical Clustering (BHC)
Bayesian Hierarchical Clustering · Also known as: BHC, probabilistic hierarchical clustering, Bayesian agglomerative clustering
Bayesian hierarchical clustering is a probabilistic agglomerative algorithm that builds a tree of nested cluster merges using Bayesian model comparison at each step. Rather than minimising a geometric linkage criterion, it evaluates at every candidate merge whether the data from two clusters are better explained by a single combined model or by two separate models, yielding a statistically principled dendrogram.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian hierarchical clustering when you need a principled, model-based dendrogram and want uncertainty quantification alongside the cluster tree rather than just a geometric hierarchy. It is well-suited to moderate-sized datasets (typically up to a few thousand observations before computational cost becomes prohibitive), to domains where a conjugate likelihood is natural — such as Gaussian continuous data or Multinomial count data — and when you want an objective stopping rule for cutting the dendrogram without arbitrary threshold choices. Do not use it as a drop-in replacement for standard hierarchical clustering on very large datasets (n > 10,000), where the O(n^2) to O(n^3) complexity makes it slow; prefer approximate or scalable variants in those settings. It is also less appropriate when the data do not fit any conjugate parametric family, or when the tree structure itself is not scientifically meaningful.
Strengths & limitations
- Replaces arbitrary linkage heuristics with statistically principled merge decisions grounded in Bayesian model comparison.
- Produces a posterior merge probability at every tree node, giving an objective criterion for selecting the number of clusters.
- Handles uncertainty in cluster structure explicitly, avoiding the false precision of a single hard partition.
- Conjugate priors yield closed-form marginal likelihoods, making exact computation feasible for Gaussian and Multinomial data.
- The resulting dendrogram is interpretable as a probabilistic model of nested group membership.
- Computational complexity is quadratic to cubic in sample size, limiting applicability to moderate datasets without approximations.
- Requires specifying a parametric likelihood and conjugate prior; misspecification of the generative model degrades clustering quality.
- The greedy merge strategy does not guarantee the globally optimal tree, so the final dendrogram may differ from the maximum-posterior tree.
- Concentration parameter alpha must be set or estimated; its choice influences the prior number of clusters and can be difficult to calibrate.
Frequently asked
How does Bayesian hierarchical clustering differ from standard hierarchical clustering?
Standard hierarchical clustering merges clusters based on a geometric distance or linkage criterion with no probabilistic interpretation. BHC replaces that criterion with a Bayesian model comparison: at each step it computes the posterior probability that two candidate clusters were generated by a single model rather than two separate models, and merges the pair with the highest such probability.
How do I choose the number of clusters from the BHC dendrogram?
Each internal node in the dendrogram carries a merge probability r_k. A natural rule is to cut the tree at the deepest node whose r_k falls below 0.5, meaning the merge was more likely than not to be spurious. This gives an objective, data-driven stopping criterion absent from classical hierarchical methods.
What likelihood and prior should I use?
For continuous data, a Gaussian likelihood with an inverse-Wishart prior on the covariance (and a Normal prior on the mean) is standard and yields a closed-form marginal likelihood. For count or categorical data, a Dirichlet-Multinomial model is the natural conjugate choice. The key requirement is that the prior be conjugate to the likelihood so the marginal integral is tractable.
Is BHC suitable for large datasets?
The naive algorithm is quadratic to cubic in n and becomes slow beyond a few thousand observations. For larger datasets, practitioners use approximations such as random-projection initialisation, sub-sampling, or divide-and-conquer BHC variants that build local trees and then merge them.
How does BHC relate to Dirichlet process mixture models?
BHC uses a Dirichlet process prior on tree partitions, so it shares the same nonparametric prior family as DP mixtures. However, DP mixtures produce a flat (exchangeable) partition posterior, whereas BHC produces a tree of nested partitions. The two methods answer different questions: DP mixtures ask how many clusters and which observations belong to each; BHC additionally asks how clusters relate hierarchically.
Sources
- Heller, K. A. & Ghahramani, Z. (2005). Bayesian hierarchical clustering. In Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), pp. 297–304. ACM. DOI: 10.1145/1102351.1102389 ↗
- Murtagh, F. & Legendre, P. (2014). Ward's hierarchical agglomerative clustering method: which algorithms implement Ward's criterion? Journal of Classification, 31(3), 274–295. DOI: 10.1007/s00357-014-9161-z ↗
How to cite this page
ScholarGate. (2026, June 3). Bayesian Hierarchical Clustering. ScholarGate. https://scholargate.app/en/statistics/bayesian-hierarchical-clustering
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Cluster AnalysisStatistics↔ compare
- Bayesian Latent Class AnalysisStatistics↔ compare
- Bayesian Mixture ModelingStatistics↔ compare
- Cluster AnalysisStatistics↔ compare
- Hierarchical ClusteringMachine learning↔ compare
- Mixture ModelingStatistics↔ compare