Log-Loss (Cross-Entropy Loss)
Also known as: Cross-Entropy Loss, Logloss
Log-loss measures the difference between predicted probabilities and actual labels, penalizing confident wrong predictions more than uncertain ones. It is a standard loss function in machine learning optimization and evaluates probabilistic classifier calibration.
Key highlights
- Evaluates probabilistic predictions, not just binary class predictions.
- Penalizes overconfident wrong predictions heavily, encouraging calibrated probabilities.
- Differentiable, making it suitable as an objective function for optimization.
- Widely used in deep learning and probabilistic models.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use log-loss when you care about the quality of predicted probabilities, not just whether the final prediction is correct. It is appropriate when you need to optimize probability estimates for downstream uses (e.g., threshold selection, cost-sensitive classification). Use log-loss when confident wrong predictions are worse than uncertain wrong predictions.
Strengths & limitations
- Evaluates probabilistic predictions, not just binary class predictions.
- Penalizes overconfident wrong predictions heavily, encouraging calibrated probabilities.
- Differentiable, making it suitable as an objective function for optimization.
- Widely used in deep learning and probabilistic models.
- Not interpretable as a probability or proportion like accuracy or F1-score.
- Sensitive to class imbalance; rare class predictions can dominate the loss.
- Can be dominated by a few very confident wrong predictions.
- Requires probability outputs, not just class predictions.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does log-loss differ from accuracy?
Accuracy only cares whether the final prediction (after thresholding at 0.5) is correct. Log-loss evaluates the quality of the probability estimates themselves. A model with 90% accuracy but poor calibration could have high log-loss. Log-loss provides finer-grained evaluation of probability quality.
Why penalize confident wrong predictions more heavily?
Confident wrong predictions indicate the model is systematically overconfident. These are dangerous because downstream decisions based on these probabilities will be misled. Conservative uncertainty is preferable to overconfident wrongness. Log-loss encourages models to express appropriate uncertainty.
Can I directly compare log-loss values across different datasets?
No, log-loss values depend on baseline difficulty (class imbalance) and the actual probability distributions. A log-loss of 0.3 on one dataset might be excellent, while 0.5 on another might also be excellent. Compare log-loss only within the same dataset, or normalize by baseline random classifier loss.
Sources
- 1.Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
- 2.Bishop, C. M. (1995). Neural Networks for Pattern Recognition. Oxford University Press.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Log-Loss (Cross-Entropy Loss). ScholarGate. https://scholargate.app/model-evaluation/log-loss