Methoden vergelijken

	Modelkalibratie	Conformal Prediction
Vakgebied	Machine learning	Machine learning
Familie	Machine learning	Machine learning
Jaar van ontstaan	2017	2005
Grondlegger	Platt; Guo et al.	Vovk, Gammerman & Shafer
Type	Post-hoc probability correction technique	Distribution-free uncertainty quantification framework
Oorspronkelijke bron	Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. International Conference on Machine Learning, 1321–1330.	Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic Learning in a Random World. Springer. ISBN: 978-0-387-00152-4
Aliassen	Classifier Calibration, Probability Calibration, Score Calibration, Model Kalibrasyonu	Conformal Inference, Conformal Risk Control, Inductive Conformal Prediction, Uyumsal Tahmin
Verwant	3	2
Samenvatting	Model calibration is a post-hoc technique that adjusts the probability outputs of a trained classifier so that predicted confidence scores match empirical outcome frequencies. A classifier is said to be perfectly calibrated if, among all predictions made with confidence p, exactly a fraction p of them are correct. Systematic miscalibration of modern deep neural networks was rigorously documented by Guo et al. (2017), who showed that networks trained with standard cross-entropy loss tend to be overconfident, and proposed temperature scaling as a simple, effective remedy.	Conformal Prediction is a distribution-free framework for constructing statistically valid prediction sets (for classification) or prediction intervals (for regression) around the output of any pre-trained machine learning model. Introduced by Vovk, Gammerman, and Shafer in their 2005 monograph, it provides a finite-sample marginal coverage guarantee — the true label falls inside the prediction set with at least 1-alpha probability — without requiring parametric assumptions about the data distribution.