Bayesian Decision Tree
Bayesian Decision Tree (Bayesian CART) · Also known as: Bayesian CART, BCART, Bayesian tree induction, probabilistic decision tree
Bayesian Decision Tree (Bayesian CART) places a prior distribution over tree structures and leaf parameters, then uses Markov chain Monte Carlo to explore the posterior distribution of trees given data. Instead of a single best tree, it produces a distribution of plausible trees whose predictions are averaged, yielding calibrated uncertainty estimates alongside point predictions.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian Decision Tree when you need uncertainty quantification alongside predictions — especially in risk-sensitive domains such as medicine, policy, or finance — and when the dataset is small enough that structural uncertainty is substantial (roughly n < 500). It is valuable when interpretability is important (individual sampled trees remain human-readable) and when you want calibrated posterior predictive intervals rather than a single point estimate. Avoid it for large datasets (n > 10 000) because MCMC over tree structures becomes computationally prohibitive; in those cases Random Forest or gradient boosting is more practical. It is also less suitable when raw predictive accuracy on a held-out benchmark is the only goal, since modern boosting methods usually outperform it on accuracy alone.
Strengths & limitations
- Provides calibrated posterior predictive intervals, quantifying structural uncertainty about the tree.
- Built-in regularisation via the tree-structure prior prevents overfitting without separate cross-validation of depth.
- Individual sampled trees remain human-readable, preserving the interpretability of standard decision trees.
- Marginalises over many plausible tree structures rather than committing to a single greedy fit.
- Well-suited to small-sample settings where a single best tree is unreliable.
- Conjugate priors on leaf parameters allow efficient marginalisation, keeping computation tractable for moderate data sizes.
- MCMC over tree space is computationally expensive; scaling to large datasets (n > 10 000) is impractical without approximations.
- Prior sensitivity: results can depend on the choice of structure prior, which requires domain knowledge to set well.
- Convergence of the Markov chain is hard to verify and may require long burn-in and thinning.
- Less widely available in standard ML libraries than CART or Random Forest, raising the implementation barrier.
Frequently asked
How does Bayesian Decision Tree differ from Random Forest?
Random Forest builds many trees by bootstrap resampling and random feature subsets, then averages them for variance reduction — it is a frequentist ensemble. Bayesian Decision Tree samples trees from a posterior distribution defined by a prior and the likelihood, producing calibrated credible intervals. Random Forest is faster and typically more accurate on large datasets; Bayesian DT is better calibrated and more transparent about structural uncertainty, particularly on small data.
What prior should I use on the tree structure?
The most common choice follows Chipman et al. (1998): the probability that a node at depth d is an internal node is alpha * (1 + d)^(-beta), with alpha around 0.95 and beta around 2. This penalises deep trees without ruling them out. Adjusting beta controls how strongly the prior favours shallow trees.
How do I know if the MCMC chain has converged?
Monitor the log marginal likelihood and tree depth across iterations for signs of mixing. Running multiple independent chains and checking that their posterior summaries agree is a practical diagnostic. Gelman-Rubin statistics on scalar summaries (e.g., tree depth) are useful. Discard at least the first 20–30% of iterations as burn-in.
Is there a relationship between Bayesian Decision Tree and BART?
Yes. BART (Bayesian Additive Regression Trees, Chipman et al. 2010) extends the single Bayesian decision tree to a sum of many small trees, each with a regularising prior. BART typically has much better predictive accuracy than a single Bayesian tree and has become the standard Bayesian tree method for regression and causal inference.
Can Bayesian Decision Tree handle missing data?
Not natively — the standard formulation assumes complete data. Missing values should be imputed before fitting, or the model extended with a missing-data model. Some implementations handle missingness via surrogate splits, as in CART, but this is not part of the original Bayesian CART specification.
Sources
- Chipman, H. A., George, E. I., & McCulloch, R. E. (1998). Bayesian CART model search. Journal of the American Statistical Association, 93(443), 935–948. DOI: 10.1080/01621459.1998.10473750 ↗
- Denison, D. G. T., Mallick, B. K., & Smith, A. F. M. (1998). A Bayesian CART algorithm. Biometrika, 85(2), 363–377. link ↗
How to cite this page
ScholarGate. (2026, June 3). Bayesian Decision Tree (Bayesian CART). ScholarGate. https://scholargate.app/en/machine-learning/bayesian-decision-tree
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Random ForestMachine learning↔ compare
- Decision TreeMachine learning↔ compare
- Gaussian ProcessMachine learning↔ compare
- Random ForestMachine learning↔ compare
- Regularized Decision TreeMachine learning↔ compare