Сравнение на методи

	Стохастична оптимизация	Байесовска оптимизация
Област	Оптимизация	Оптимизация
Семейство	Process / pipeline	Process / pipeline
Година на възникване	1951 (SGD); 2014 (Adam)	1975 (foundational); 2012 (ML standard)
Създател	—	Mockus (1975); popularised for ML by Snoek, Larochelle & Adams (2012)
Тип	Gradient-based iterative optimization	Sequential model-based black-box optimization
Основополагащ източник	Robbins, H. & Monro, S. (1951). A Stochastic Approximation Method. Annals of Mathematical Statistics, 22(3), 400-407.	Snoek, J., Larochelle, H., & Adams, R.P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. Advances in Neural Information Processing Systems (NeurIPS), 25.
Други названия	Stokastik Optimizasyon (SGD & Varyantları), stochastic gradient descent, SGD, Adam	Bayesçi Optimizasyon (Hyperparameter Tuning), surrogate-based optimization, sequential model-based optimization, SMBO
Свързани	3	2
Резюме	Stochastic optimization is a family of iterative methods that minimize an objective function by computing gradients on randomly sampled subsets of data — mini-batches — rather than on the entire dataset at once. Pioneered by Robbins and Monro in 1951 as stochastic approximation, the approach became the standard engine for training large-scale machine-learning models through variants such as SGD with momentum, AdaGrad, RMSProp, and Adam.	Bayesian Optimization is a sequential, model-based strategy for finding the optimum of expensive black-box functions with as few evaluations as possible. Rooted in the work of Mockus (1975) and brought to mainstream machine-learning practice by Snoek, Larochelle, and Adams (2012), it fits a probabilistic surrogate model — typically a Gaussian Process — to past observations and uses an acquisition function to decide where to probe next, balancing exploration of unknown regions with exploitation of promising ones.