Decision Tree
Also known as: Karar Ağacı (Decision Tree), karar ağacı, classification tree, regression tree, CART
A Decision Tree is an interpretable classification and regression method, formalised by Breiman, Friedman, Olshen and Stone in their 1984 CART framework, that partitions the data with hierarchical if-then rules. Each split sends observations down one branch or another until a prediction is read off the leaf.
Key highlights
- Highly interpretable: the path from root to leaf is a plain if-then rule.
- No feature scaling required; handles continuous, categorical, binary and ordinal features directly.
- Low difficulty — suitable even for beginner users.
- Serves as the building block of ensembles such as Random Forest and Gradient Boosting.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use a decision tree when you want an interpretable model for classification, prediction or explanation on cross-sectional tabular data with at least about 30 observations, mixing continuous, categorical, binary and ordinal features. No feature scaling is needed and the rules are easy to communicate. Below roughly 30 cases the tree overfits readily — leaves can hold a single observation — and a simpler, more stable model such as logistic regression or naive Bayes is safer.
Strengths & limitations
- Highly interpretable: the path from root to leaf is a plain if-then rule.
- No feature scaling required; handles continuous, categorical, binary and ordinal features directly.
- Low difficulty — suitable even for beginner users.
- Serves as the building block of ensembles such as Random Forest and Gradient Boosting.
- Prone to overfitting unless pruned via max_depth or min_samples_leaf.
- On small samples (n below about 30) the tree overfits easily, with leaves that may contain a single observation.
- After pruning a small-sample tree may retain too few splits to be useful, where logistic regression is more stable.
- A single tree is unstable — small data changes can produce a very different structure.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How is a split chosen?
The CART algorithm evaluates candidate splits across all features and thresholds and picks the one that most reduces node impurity, commonly measured by the Gini index or information gain for classification.
Why does a decision tree overfit, and how do I stop it?
Left to grow freely a tree keeps splitting until leaves are nearly pure, memorising noise. Restrain it by limiting depth (max_depth), requiring a minimum number of cases per leaf (min_samples_leaf), or pruning back branches that add little.
How much data do I need?
Aim for at least about 30 observations. Below that the tree overfits easily — leaves can contain a single case — and a simpler model such as logistic regression or naive Bayes is more stable.
How does a decision tree relate to Random Forest?
A single decision tree is the building block of ensembles: Random Forest grows many trees on resampled data and combines their votes, and Gradient Boosting builds trees sequentially — both trading the single tree's transparency for greater accuracy and stability.
Sources
- 1.Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984). Classification and Regression Trees. Wadsworth.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 1). Decision Tree. ScholarGate. https://scholargate.app/machine-learning/decision-tree