방법 비교

	확률적 최적화	베이지안 최적화	진화 전략 (CMA-ES)	강건 최적화
분야	최적화	최적화	최적화	최적화
계열	Process / pipeline	Process / pipeline	Process / pipeline	Process / pipeline
기원 연도	1951 (SGD); 2014 (Adam)	1975 (foundational); 2012 (ML standard)	2001	1970s theoretical roots; modern tractable form from late 1990s–2004
창시자	—	Mockus (1975); popularised for ML by Snoek, Larochelle & Adams (2012)	Nikolaus Hansen & Andreas Ostermeier	Ben-Tal, El Ghaoui & Nemirovski (seminal book, 2009); Bertsimas & Sim (tractable polyhedral formulation, 2004)
유형	Gradient-based iterative optimization	Sequential model-based black-box optimization	Derivative-free continuous black-box optimizer	Mathematical programming framework
원전	Robbins, H. & Monro, S. (1951). A Stochastic Approximation Method. Annals of Mathematical Statistics, 22(3), 400-407.	Snoek, J., Larochelle, H., & Adams, R.P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. Advances in Neural Information Processing Systems (NeurIPS), 25.	Hansen, N. & Ostermeier, A. (2001). Completely Derandomized Self-Adaptation in Evolutionary Strategies. Evolutionary Computation, 9(2), 159-195.	Ben-Tal, A., El Ghaoui, L. & Nemirovski, A. (2009). Robust Optimization. Princeton University Press. ISBN: 9780691143682
별칭	Stokastik Optimizasyon (SGD & Varyantları), stochastic gradient descent, SGD, Adam	Bayesçi Optimizasyon (Hyperparameter Tuning), surrogate-based optimization, sequential model-based optimization, SMBO	CMA-ES, Evolution Strategy, Evrimsel Strateji (CMA-ES), self-adapting evolution strategy	minimax optimization, worst-case optimization, Gürbüz Optimizasyon (Robust Optimization)
관련	3	2	5	5
요약	Stochastic optimization is a family of iterative methods that minimize an objective function by computing gradients on randomly sampled subsets of data — mini-batches — rather than on the entire dataset at once. Pioneered by Robbins and Monro in 1951 as stochastic approximation, the approach became the standard engine for training large-scale machine-learning models through variants such as SGD with momentum, AdaGrad, RMSProp, and Adam.	Bayesian Optimization is a sequential, model-based strategy for finding the optimum of expensive black-box functions with as few evaluations as possible. Rooted in the work of Mockus (1975) and brought to mainstream machine-learning practice by Snoek, Larochelle, and Adams (2012), it fits a probabilistic surrogate model — typically a Gaussian Process — to past observations and uses an acquisition function to decide where to probe next, balancing exploration of unknown regions with exploitation of promising ones.	CMA-ES, short for Covariance Matrix Adaptation Evolution Strategy, is a modern derivative-free optimizer for continuous black-box functions introduced by Hansen and Ostermeier in 2001. It maintains a population of candidate solutions drawn from a multivariate normal distribution and iteratively updates the distribution's mean, step size, and full covariance matrix to steer the search toward better regions of the parameter space. It has become the de-facto standard for continuous black-box optimization and is widely used in neural architecture search and reinforcement-learning policy optimization.	Robust optimization is a mathematical programming framework, formalised by Ben-Tal and Nemirovski in the late 1990s and made broadly tractable by Bertsimas and Sim (2004), that finds decisions guaranteed to perform acceptably under every scenario within a predefined uncertainty set — rather than assuming parameter values are known exactly. Instead of optimising for a single expected outcome, it minimises the worst-case objective across all plausible realisations of uncertain data.