Decision Tree
Decision Tree (CART — Classification and Regression Trees) · Also known as: Karar Ağacı (Decision Tree), karar ağacı, classification tree, regression tree, CART
A Decision Tree is an interpretable classification and regression method, formalised by Breiman, Friedman, Olshen and Stone in their 1984 CART framework, that partitions the data with hierarchical if-then rules. Each split sends observations down one branch or another until a prediction is read off the leaf.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+33 more
When to use it
Use a decision tree when you want an interpretable model for classification, prediction or explanation on cross-sectional tabular data with at least about 30 observations, mixing continuous, categorical, binary and ordinal features. No feature scaling is needed and the rules are easy to communicate. Below roughly 30 cases the tree overfits readily — leaves can hold a single observation — and a simpler, more stable model such as logistic regression or naive Bayes is safer.
Strengths & limitations
- Highly interpretable: the path from root to leaf is a plain if-then rule.
- No feature scaling required; handles continuous, categorical, binary and ordinal features directly.
- Low difficulty — suitable even for beginner users.
- Serves as the building block of ensembles such as Random Forest and Gradient Boosting.
- Prone to overfitting unless pruned via max_depth or min_samples_leaf.
- On small samples (n below about 30) the tree overfits easily, with leaves that may contain a single observation.
- After pruning a small-sample tree may retain too few splits to be useful, where logistic regression is more stable.
- A single tree is unstable — small data changes can produce a very different structure.
Frequently asked
How is a split chosen?
The CART algorithm evaluates candidate splits across all features and thresholds and picks the one that most reduces node impurity, commonly measured by the Gini index or information gain for classification.
Why does a decision tree overfit, and how do I stop it?
Left to grow freely a tree keeps splitting until leaves are nearly pure, memorising noise. Restrain it by limiting depth (max_depth), requiring a minimum number of cases per leaf (min_samples_leaf), or pruning back branches that add little.
How much data do I need?
Aim for at least about 30 observations. Below that the tree overfits easily — leaves can contain a single case — and a simpler model such as logistic regression or naive Bayes is more stable.
How does a decision tree relate to Random Forest?
A single decision tree is the building block of ensembles: Random Forest grows many trees on resampled data and combines their votes, and Gradient Boosting builds trees sequentially — both trading the single tree's transparency for greater accuracy and stability.
Sources
- Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984). Classification and Regression Trees. Wadsworth. DOI: 10.1201/9781315139470 ↗
How to cite this page
ScholarGate. (2026, June 1). Decision Tree (CART — Classification and Regression Trees). ScholarGate. https://scholargate.app/en/machine-learning/decision-tree
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Logistic RegressionResearch Statistics↔ compare
- Naive BayesMachine learning↔ compare
- Random ForestMachine learning↔ compare
- Support Vector MachineMachine learning↔ compare
- XGBoostMachine learning↔ compare