Hamilton-Jacobi-Bellman Equation
Also known as: HJB Equation, Bellman Equation, Dynamic Programming
The Hamilton-Jacobi-Bellman (HJB) equation is a partial differential equation characterizing the optimal cost-to-go function in dynamic programming. Developed by Bellman in 1957, HJB provides both necessary and sufficient conditions for optimality, enabling elegant theoretical analysis and numerical solutions for optimal control problems. HJB is fundamental to reinforcement learning, approximate dynamic programming, and real-time control.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use HJB for offline optimal control synthesis when sufficiency and theoretical guarantees are important. It is ideal for nonlinear systems, time-optimal problems, and games. HJB excels in analysis; for computation, numerical schemes can be expensive. Use HJB when system structure is simple (low dimension) or when you need the value function for learning and adaptive control.
Strengths & limitations
- Provides necessary and sufficient optimality conditions; solutions are guaranteed optimal (not just critical points).
- Works for nonlinear, time-varying, and constrained systems.
- Value function V(x,t) encodes all information about optimal behavior; useful for analyzing sensitivity.
- Extends naturally to stochastic control and differential games.
- Foundation for reinforcement learning and approximate dynamic programming.
- Solving HJB is computationally expensive; the PDE curse of dimensionality (memory/computation grows exponentially in state dimension).
- Analytical solutions rare; numerical PDE solvers are complex and may be inaccurate.
- Verification of solutions is difficult; checking sufficiency conditions numerically is unreliable.
- No direct guidance on numerical method selection; problem-dependent.
- Terminal condition or value function structure must be specified; if omitted, solution quality degrades.
Frequently asked
What is the curse of dimensionality in HJB?
HJB is a PDE in state space; discretizing an n-dimensional state space with m grid points requires m^n grid cells. For n=10, m=100, this is 10^20 cells—infeasible! This is the curse of dimensionality. Mitigation: use sparse grids, function approximation (neural networks), or restrict to low-dimensional subspaces.
Sources
How to cite this page
ScholarGate. (2026, June 3). Hamilton-Jacobi-Bellman Equation. ScholarGate. https://scholargate.app/en/control-theory/hamilton-jacobi-bellman-equation
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Linear Quadratic RegulatorControl Theory↔ compare
- Model Predictive ControlControl Theory↔ compare
- Pontryagin Maximum PrincipleControl Theory↔ compare