LightGBM
Also known as: LightGBM, Light Gradient Boosting Machine, lgbm, leaf-wise gradient boosting
LightGBM is Microsoft's gradient boosting decision tree implementation, introduced by Ke and colleagues in 2017, that grows trees leaf-wise and bins features into histograms for speed. On large datasets it is much faster than XGBoost while retaining strong predictive accuracy.
Key highlights
- Much faster than XGBoost on large datasets thanks to histogram-based splitting.
- Leaf-wise growth converges quickly and often reaches high accuracy.
- Handles categorical features directly, without manual one-hot encoding.
- Works for prediction, classification, and forecasting across cross-sectional, panel, and time-series data.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
A strong choice for prediction, classification, and forecasting on large tabular datasets — cross-sectional, panel, or time-series — with continuous, categorical, binary, or ordinal features and at least about 200 observations. It is most valuable when training speed and scale matter and you can tune regularisation. Below roughly 200 observations the leaf-wise growth tends to overfit badly, and a level-wise method such as XGBoost or a more stable Random Forest is safer.
Strengths & limitations
- Much faster than XGBoost on large datasets thanks to histogram-based splitting.
- Leaf-wise growth converges quickly and often reaches high accuracy.
- Handles categorical features directly, without manual one-hot encoding.
- Works for prediction, classification, and forecasting across cross-sectional, panel, and time-series data.
- Leaf-wise growth overfits on small samples unless carefully regularised.
- Requires tuning num_leaves and min_data_in_leaf to control complexity.
- On datasets below about 200 observations a level-wise method (XGBoost) or Random Forest is more reliable.
- Like other boosted trees, it is a black box with no directly interpretable coefficients.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How is LightGBM different from XGBoost?
Both are gradient boosting decision tree libraries. LightGBM uses histogram-based splitting and leaf-wise (best-first) tree growth, which makes it much faster on large datasets. XGBoost grows level-wise, which is often safer on smaller samples.
Why does LightGBM overfit on small datasets?
Its leaf-wise growth always splits the leaf with the largest gain, which can carve the data into too many leaves relative to its size. On small samples (below about 200 observations) this leads to serious overfitting, so a level-wise method or Random Forest is recommended.
Do I need to one-hot encode categorical variables?
No. LightGBM can take categorical features directly, so manual one-hot encoding is usually unnecessary and can even hurt performance.
Which settings control overfitting?
The number of leaves (num_leaves) and the minimum data per leaf (min_data_in_leaf) are the key regularisation controls; tightening them limits tree complexity relative to the dataset size.
Sources
- 1.Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q. & Liu, T.-Y. (2017). LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Advances in Neural Information Processing Systems (NeurIPS) 30, 3146–3154.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 1). LightGBM. ScholarGate. https://scholargate.app/machine-learning/lightgbm