Vertaile menetelmiä

	Epäkäsitteellinen optimointi	Konveksi optimointi	Dynaaminen ohjelmointi	Stokastinen optimointi
Tieteenala	Optimointi	Optimointi	Optimointi	Optimointi
Menetelmäperhe	Process / pipeline	Process / pipeline	Process / pipeline	Process / pipeline
Syntyvuosi	2006	2004	1957	1951 (SGD); 2014 (Adam)
Kehittäjä	Jorge Nocedal & Stephen Wright	Stephen Boyd & Lieven Vandenberghe	Richard Bellman	—
Tyyppi	Continuous mathematical optimization	Mathematical optimization framework	Exact combinatorial optimization via recursive decomposition	Gradient-based iterative optimization
Alkuperäislähde	Nocedal, J., & Wright, S. J. (2006). Numerical Optimization (2nd ed.). Springer. ISBN: 978-0-387-30303-1	Boyd, S., & Vandenberghe, L. (2004). Convex Optimization. Cambridge University Press. ISBN: 978-0-521-83378-3	Bellman, R. (1957). Dynamic Programming. Princeton University Press. ISBN: 978-0-691-07951-6	Robbins, H. & Monro, S. (1951). A Stochastic Approximation Method. Annals of Mathematical Statistics, 22(3), 400-407.
Rinnakkaisnimet	NLP optimization, Constrained nonlinear optimization, Smooth optimization, Doğrusal olmayan programlama	Convex Programming, Disciplined Convex Programming, Dışbükey Optimizasyon, Convex Mathematical Programming	DP, Bellman's Principle of Optimality, Recursive Optimization, Dinamik Programlama	Stokastik Optimizasyon (SGD & Varyantları), stochastic gradient descent, SGD, Adam
Liittyvät	3	3	3	3
Tiivistelmä	Nonlinear programming (NLP) is a branch of mathematical optimization concerned with problems in which the objective function or at least one constraint is nonlinear. Formalized comprehensively by Jorge Nocedal and Stephen Wright in their seminal 2006 text, NLP encompasses gradient-based algorithms — including sequential quadratic programming (SQP), interior-point methods, and quasi-Newton approaches — for finding locally or globally optimal solutions to continuous decision problems arising across engineering, economics, and the physical sciences.	Convex optimization is a subfield of mathematical optimization that studies the problem of minimizing convex functions over convex sets. Formalized and popularized by Stephen Boyd and Lieven Vandenberghe in their landmark 2004 textbook, the framework unifies a wide family of problems — including linear programming, quadratic programming, semidefinite programming, and second-order cone programming — under a single theoretical roof. Its defining property is that any locally optimal solution is also globally optimal, making it tractable and reliable for engineering, statistics, machine learning, and operations research.	Dynamic Programming (DP) is an exact optimization technique introduced by Richard Bellman in 1957 for solving multi-stage decision problems. It decomposes a complex problem into simpler, overlapping subproblems, solves each subproblem once, and stores the results to avoid redundant computation. Grounded in the Principle of Optimality, DP guarantees globally optimal solutions whenever the problem exhibits overlapping subproblems and optimal substructure.	Stochastic optimization is a family of iterative methods that minimize an objective function by computing gradients on randomly sampled subsets of data — mini-batches — rather than on the entire dataset at once. Pioneered by Robbins and Monro in 1951 as stochastic approximation, the approach became the standard engine for training large-scale machine-learning models through variants such as SGD with momentum, AdaGrad, RMSProp, and Adam.