Bayesian Random Forest
Bayesian Random Forest (Bayesian Ensemble of Decision Trees) · Also known as: Bayesian Forest, BRF, Empirical Bayesian Forest, posterior random forest
Bayesian Random Forest extends the classical random forest by placing a prior distribution over tree structures and leaf parameters, then sampling or approximating the posterior over that ensemble. The result is a set of predictions accompanied by calibrated uncertainty estimates — a capability standard random forests lack — making it valuable when knowing how confident the model is matters as much as the prediction itself.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use Bayesian random forest when you need both accurate predictions and reliable uncertainty quantification — for example in medical diagnosis, financial risk, or safety-critical applications where a model must flag its own low-confidence predictions. It suits tabular data with moderate to large sample sizes (roughly 100+ observations) and nonlinear feature relationships. Prefer it over a standard random forest when miscalibrated confidence is costly. Do not use it when interpretability in the form of explicit coefficients is required, when computational budget is very tight (posterior sampling is slower than standard forest training), or when the dataset is tiny (fewer than ~50 observations), where the posterior becomes dominated by the prior and results are unreliable.
Strengths & limitations
- Provides calibrated predictive uncertainty, not just point estimates.
- Inherits the strong predictive accuracy and robustness of standard random forests.
- Posterior weighting regularises the ensemble, reducing overfitting compared to equally-weighted bagging.
- Handles nonlinear interactions and mixed feature types without distributional assumptions.
- Uncertainty estimates can be used to guide active learning or flag high-risk predictions downstream.
- Substantially more computationally expensive than standard random forests, especially with full MCMC sampling.
- Posterior approximations (empirical Bayes) sacrifice some calibration accuracy for tractability.
- No explicit coefficients; the model remains a black box from an inference standpoint.
- Requires careful prior specification — poorly chosen priors can dominate inference on small datasets.
Frequently asked
How does a Bayesian random forest differ from a standard random forest?
A standard random forest averages predictions from equally-weighted trees grown by a deterministic bootstrap procedure. A Bayesian random forest weights trees by their posterior probability given the data, and the spread across trees yields calibrated uncertainty intervals rather than a single point estimate.
Is it much slower than a standard random forest?
Yes. Full MCMC-based inference can be orders of magnitude slower. Empirical Bayesian approximations (e.g., Taddy et al. 2015) are closer to standard forest training speed but still add overhead. The additional cost is the price of obtaining reliable uncertainty estimates.
Do I need to set a prior, and how sensitive are results to it?
Yes, a prior over tree structure and leaf parameters must be specified. On large datasets the likelihood dominates and the prior matters less. On small datasets the prior can dominate, so it should reflect genuine domain beliefs about plausible model complexity.
Can I use it for classification as well as regression?
Yes. For classification the posterior predictive gives class probabilities with associated uncertainty; for regression it gives a posterior mean and credible interval. Both use cases are supported in implementations such as the R package bayesianForest and Python-based variants.
When should I prefer BART over a Bayesian random forest?
Bayesian Additive Regression Trees (BART) is a closely related fully Bayesian tree ensemble with well-studied software (the R package BART). Prefer BART when you want a thoroughly validated Bayesian implementation with established software support. Bayesian random forests may be preferred when you need an online or scalable variant (e.g., Mondrian forests).
Sources
- Taddy, M., Chen, C., Yu, J., & Wyle, M. (2015). Bayesian and Empirical Bayesian Forests. Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), PMLR 37, 967–976. link ↗
- Lakshminarayanan, B., Roy, D. M., & Teh, Y. W. (2016). Mondrian Forests for Large-Scale Regression when Uncertainty Matters. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (AISTATS 2016), PMLR 51, 1478–1487. link ↗
How to cite this page
ScholarGate. (2026, June 3). Bayesian Random Forest (Bayesian Ensemble of Decision Trees). ScholarGate. https://scholargate.app/en/machine-learning/bayesian-random-forest
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Bayesian Active LearningMachine learning↔ compare
- Bayesian Decision TreeMachine learning↔ compare
- Bayesian Semi-supervised LearningMachine learning↔ compare
- Gaussian ProcessMachine learning↔ compare
- Random ForestMachine learning↔ compare