Augmented Lagrangian Method
Augmented Lagrangian Method for Constrained Optimization · Also known as: method of multipliers, augmented Lagrangian, ADMM
The Augmented Lagrangian Method, developed by Magnus R. Hestenes and M. J. D. Powell in 1969, is a powerful technique for solving constrained optimization problems. It converts a constrained problem into a sequence of unconstrained subproblems by augmenting the Lagrangian with a quadratic penalty term, enabling efficient solution of large-scale problems including convex and nonconvex cases.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Apply the augmented Lagrangian method to constrained optimization problems where decomposition into subproblems is possible or when constraints are difficult to handle directly. It is ideal for large-scale problems with special structure, convex or nonconvex objectives, and equality or inequality constraints. Use it when subproblem solution is easier than solving the full problem. For unconstrained problems or very large-scale problems requiring parallelization, consider specialized variants like ADMM.
Strengths & limitations
- Enables solution of difficult constrained problems through manageable subproblems
- Handles both equality and inequality constraints naturally
- Works for nonconvex problems with appropriate modifications
- Avoids ill-conditioning issues of pure penalty methods
- Naturally amenable to parallel and distributed computing
- Flexible in choice of subproblem solver
- Requires solving subproblems accurately, which can be expensive
- Convergence speed depends on subproblem solver quality
- Penalty parameter selection and updates require tuning
- Less effective than specialized solvers on problems without exploitable structure
- Can be sensitive to numerical precision in constraint residual calculations
Frequently asked
How does the augmented Lagrangian improve upon classical Lagrangian relaxation?
Classical Lagrangian methods require penalty weights to infinity for constraint satisfaction, causing ill-conditioning. Augmented Lagrangian maintains reasonable penalty weights and updates multipliers adaptively, achieving faster convergence and better numerical stability.
What is the relationship between augmented Lagrangian and ADMM?
ADMM is a specialized variant of the augmented Lagrangian method designed for convex problems with two-block separable structure. It decomposes problems into two subproblems solved alternately, enabling parallel and distributed implementation.
How should the penalty parameter be chosen and updated?
Start with a moderate penalty parameter (e.g., 1 or based on problem scaling). Increase it if constraints are violated after updates; decrease if subproblem solutions become difficult. Adaptive schemes adjust it based on constraint residual trends.
Can the method solve nonconvex problems?
Yes, with appropriate modifications. Convergence guarantees differ (convergence to stationary points rather than global optima), and subproblem solvers must handle nonconvexity. The method remains effective for many practical nonconvex problems.
Sources
- Hestenes, M. R. (1969). Multiplier and gradient methods. Journal of Optimization Theory and Applications, 4(5), 303-320. DOI: 10.1007/BF00927673 ↗
- Powell, M. J. D. (1969). A method for nonlinear constraints in minimization problems. In Optimization (pp. 283-298). Academic Press. link ↗
- Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1-122. DOI: 10.1561/2200000016 ↗
How to cite this page
ScholarGate. (2026, June 3). Augmented Lagrangian Method for Constrained Optimization. ScholarGate. https://scholargate.app/en/operations-research/augmented-lagrangian-method
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Benders DecompositionOperations Research↔ compare
- Column Generation (Dantzig-Wolfe)Operations Research↔ compare
- Simplex MethodOperations Research↔ compare