XGBoost
Also known as: XGBoost, extreme gradient boosting, scalable tree boosting
XGBoost (Extreme Gradient Boosting) is a scalable tree-boosting algorithm introduced by Tianqi Chen and Carlos Guestrin in 2016. It builds a strong predictor by adding decision trees one at a time, each correcting the errors left by the trees before it, and is a powerful prediction method widely used in competitions.
Key highlights
- Strong predictive accuracy on tabular data, which is why it is so common in competitions.
- Built-in regularisation penalises tree complexity and helps control overfitting.
- Tolerates missing values and handles mixed feature types (continuous, categorical, binary, count).
- Scales well and benefits directly from feature engineering.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use XGBoost for accurate classification or prediction on tabular data with continuous, categorical, binary, or count features and at least about 100 observations, when predictive performance is the priority. It does not require normally distributed data; sufficient data and thoughtful feature engineering are the main practical assumptions. With fewer than ~100 cases, gradient boosting tends to overfit and a simpler model (such as logistic regression or a single decision tree) is more dependable.
Strengths & limitations
- Strong predictive accuracy on tabular data, which is why it is so common in competitions.
- Built-in regularisation penalises tree complexity and helps control overfitting.
- Tolerates missing values and handles mixed feature types (continuous, categorical, binary, count).
- Scales well and benefits directly from feature engineering.
- Needs enough data — roughly 100+ observations — or it overfits and cross-validation results become unreliable.
- On small samples there is not enough data to tune the many hyperparameters, so a simpler model generalises better.
- Many hyperparameters and a black-box ensemble make it harder to set up and interpret than a single model.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How much data does XGBoost need?
Around 100 observations is a sensible minimum. With fewer than 100 cases gradient boosting tends to overfit and cross-validation results become unreliable, so a simpler model such as logistic regression or a single decision tree is more dependable.
Does XGBoost require normally distributed data?
No. It makes no normality assumption and works with continuous, categorical, binary, and count features. The main practical requirements are having enough data and doing some thoughtful feature engineering.
How is XGBoost different from Random Forest?
Random Forest grows trees independently and averages them to reduce variance. XGBoost grows trees sequentially, each one correcting the errors of the previous ones, and adds a regularisation penalty on tree complexity, which can reach higher accuracy with careful tuning.
How do I interpret an XGBoost model?
Since the model is an ensemble of many trees, contributions are usually examined with feature-attribution analysis such as SHAP, alongside test-set metrics like RMSE or AUC and cross-validated hyperparameter tuning.
Sources
- 1.Chen, T. & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD, 785–794.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 1). XGBoost. ScholarGate. https://scholargate.app/machine-learning/xgboost