Optimal Control
Optimal control determines the control inputs that steer a dynamical system so as to optimize a performance criterion over time.
Definition
An optimal control problem seeks a control function that minimizes a cost functional subject to differential equations governing the state; its solution is characterized by necessary conditions from the maximum principle or by the value function of dynamic programming.
Scope
This topic covers the formulation of control problems with state dynamics and cost functionals, Pontryagin's maximum principle and the adjoint equations, dynamic programming and the Hamilton-Jacobi-Bellman equation, the linear-quadratic regulator, and the relationship to the classical calculus of variations.
Core questions
- What control law minimizes a given cost over the system's trajectory?
- What necessary conditions must an optimal control satisfy?
- How does dynamic programming characterize the optimal value function?
- How does optimal control extend the calculus of variations to constrained inputs?
Key theories
- Pontryagin maximum principle
- An optimal control maximizes a Hamiltonian at each instant, with an adjoint costate variable evolving backward in time, giving necessary conditions even when controls are constrained.
- Dynamic programming and the HJB equation
- Bellman's principle of optimality leads to the Hamilton-Jacobi-Bellman partial differential equation for the value function, whose solution yields the optimal feedback control.
- Linear-quadratic regulator
- For linear dynamics and quadratic cost, the optimal control is a linear state feedback determined by the solution of a Riccati equation, a cornerstone of control engineering.
Clinical relevance
Optimal control governs the guidance of aircraft and spacecraft, process and robotics control, economic and resource planning over time, and treatment-scheduling models, providing the principled way to act optimally on a dynamical system.
History
Optimal control emerged in the 1950s from the calculus of variations under the pressure of aerospace problems. Pontryagin and his coworkers established the maximum principle around 1956-1962, Bellman developed dynamic programming in parallel, and Kalman's linear-quadratic and filtering theory made the subject central to modern engineering.
Key figures
- Lev Pontryagin
- Richard Bellman
- Rudolf Kalman
- Constantin Caratheodory
Related topics
Seminal works
- pontryagin1962
- bertsekas2017
- liberzon2012
Frequently asked questions
- How does optimal control relate to the calculus of variations?
- The calculus of variations optimizes over curves freely, while optimal control optimizes over the inputs to a dynamical system, often with constraints on the controls. The maximum principle generalizes the classical Euler-Lagrange conditions to this constrained, system-driven setting.
- What is the difference between the maximum principle and dynamic programming?
- The maximum principle gives necessary conditions along a single optimal trajectory using an adjoint variable, whereas dynamic programming characterizes the optimal cost from every state through the Hamilton-Jacobi-Bellman equation, yielding a feedback law. The two viewpoints are complementary and connected.