Ensemble Decision Tree
Ensemble Decision Tree (Combined Decision Tree Classifiers and Regressors) · Also known as: decision tree ensemble, ensemble of decision trees, combined decision trees, multiple classifier system (decision trees)
Ensemble Decision Tree methods train multiple decision trees and combine their outputs to produce predictions that are more accurate and stable than any single tree. Covering strategies such as bagging, random subspacing, and voting, they are among the most effective off-the-shelf techniques for tabular classification and regression tasks.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use ensemble decision trees when you need reliable predictive performance on tabular data with possibly nonlinear relationships, feature interactions, or a mixture of continuous and categorical variables, and when interpretability at the level of individual coefficients is not required. They are well-suited to medium and large datasets (roughly 100 or more observations). Do not use them as a first choice when you need a fully transparent, auditable model (a single decision tree or logistic regression is better for that), when the dataset is very small (fewer than 50 observations, where overfitting risk increases), or when inference about individual predictors' causal effects is the primary goal.
Strengths & limitations
- Substantially higher accuracy and stability than a single decision tree on most tabular tasks.
- Handles mixed feature types (continuous, categorical, ordinal, binary) without preprocessing assumptions.
- Naturally captures nonlinear relationships and feature interactions without explicit engineering.
- Provides feature importance rankings that help identify the most predictive variables.
- Robust to noisy labels and outliers due to the averaging effect across many trees.
- Loss of the single-tree transparency: the ensemble cannot be visualised as one interpretable tree structure.
- Higher computational cost at training and prediction time compared with a single tree, especially for large ensembles.
- Feature importance measures can be biased toward high-cardinality features and may be unstable on small datasets.
- Combining trees by simple averaging or voting does not guarantee optimal calibration of predicted probabilities.
Frequently asked
How is an ensemble decision tree different from a single decision tree?
A single tree is grown to fit the training data and produces one deterministic prediction. An ensemble grows many trees on varied views of the data and combines their predictions. The combination reduces variance and overfitting, typically improving generalisation substantially.
What is the difference between this and Random Forest?
Random Forest is the most popular specific instance of an ensemble decision tree method, combining bootstrap sampling of observations with random feature subsampling at each split. Ensemble decision tree is the broader category that also includes methods using only bootstrap (bagging), only feature subsampling, or other diversity strategies.
How many trees should I use?
Plot out-of-bag error or cross-validated error against the number of trees and stop adding trees once the curve flattens. A practical starting point is 100 trees; for high-dimensional data, 500 or more may be needed.
Does an ensemble decision tree require feature scaling?
No. Decision trees split on threshold values and are invariant to monotonic transformations of features, so normalisation or standardisation is not required before training.
Can ensemble decision trees handle missing data?
Many implementations support surrogate splits or can pass missing values to the majority branch, offering some native tolerance. However, explicit imputation before training generally improves performance and should be preferred when feasible.
Sources
- Dietterich, T. G. (2000). Ensemble methods in machine learning. In Multiple Classifier Systems, Lecture Notes in Computer Science, vol. 1857, pp. 1–15. Springer, Berlin, Heidelberg. DOI: 10.1007/3-540-45014-9_1 ↗
- Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140. DOI: 10.1007/BF00058655 ↗
How to cite this page
ScholarGate. (2026, June 3). Ensemble Decision Tree (Combined Decision Tree Classifiers and Regressors). ScholarGate. https://scholargate.app/en/machine-learning/ensemble-decision-tree
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- BaggingMachine learning↔ compare
- BoostingMachine learning↔ compare
- Decision TreeMachine learning↔ compare
- Extra TreesMachine learning↔ compare
- Random ForestMachine learning↔ compare
- Voting EnsembleMachine learning↔ compare