GMRES
Also known as: GMRES(m), restarted GMRES, Krylov-GMRES
GMRES (Generalized Minimal Residual) is an iterative method for solving large sparse non-symmetric or nonsymmetric linear systems Ax = b, developed by Saad and Schultz in 1986. It builds an orthonormal Krylov basis using Arnoldi's method and solves a least-squares problem to minimize residual at each iteration.
Key highlights
- Handles arbitrary non-symmetric and complex matrices; makes no positivity assumptions
- Monotonic reduction in residual norm guarantees progress; residual history is reliable
- Flexible preconditioning available (FGMRES); works with left or right preconditioners
- Proven convergence: eventually (possibly after many iterations) reaches any prescribed tolerance
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use GMRES for non-symmetric or ill-conditioned systems where CG is not applicable. Essential for advection-dominated PDEs, coupled multiphysics problems, and indefinite matrices. Restart parameter m = 30–40 balances memory and convergence. For symmetric matrices, CG or MINRES are more efficient.
Strengths & limitations
- Handles arbitrary non-symmetric and complex matrices; makes no positivity assumptions
- Monotonic reduction in residual norm guarantees progress; residual history is reliable
- Flexible preconditioning available (FGMRES); works with left or right preconditioners
- Proven convergence: eventually (possibly after many iterations) reaches any prescribed tolerance
- Full GMRES requires storing m Krylov vectors; memory demand scales as O(m·n), making it expensive for large m
- Cost per iteration grows as O(k·n) for iteration k; restarting every m steps mitigates but incurs convergence penalty
- Without preconditioning, convergence is often slow for ill-conditioned problems
- Householder or Givens orthogonalization required for stability; adds computational overhead
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is the Krylov subspace and why does Arnoldi orthogonalization matter?
The Krylov subspace K_k(A, r₀) = span{r₀, A r₀, A² r₀, …, A^{k-1} r₀} contains increasingly rich information about A. Arnoldi orthogonalization produces an orthonormal basis for this space, enabling stable least-squares minimization of residuals.
Why restart at all if full GMRES converges?
Full GMRES stores all k Krylov vectors, demanding O(k·n) memory and O(k²) operations per iteration. After m = 30–50 iterations, memory exceeds availability. Restarting discards old vectors and restarts the process, trading some convergence speed for feasibility on large systems.
What restart parameter m should I choose?
m = 30 is a practical default; increase to 50–100 if memory allows and convergence is slow, decrease to 10–20 if memory is tight. Problem-dependent: elliptic PDEs often converge in 10–20 iterations; advection-dominated or coupled systems may need 40–100.
How does left preconditioning differ from right preconditioning in GMRES?
Left preconditioning solves M⁻¹ A x = M⁻¹ b, improving conditioning of M⁻¹ A. Right preconditioning solves A M⁻¹ y = b with x = M⁻¹ y, leaving the spectrum of A M⁻¹ unchanged. Left improves convergence rate; right avoids applying M⁻¹ at the end. FGMRES allows flexible (variable) preconditioners in right positioning.
Sources
- 1.Saad, Y., & Schultz, M. H. (1986). GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM Journal on Scientific and Statistical Computing, 7(3), 856–869.
- 2.Walker, H. F. (1988). Implementation of the GMRES method using Householder reflections. SIAM Journal on Scientific and Statistical Computing, 9(1), 152–163.
- 3.Saad, Y. (2003). Iterative Methods for Sparse Linear Systems (2nd ed.). SIAM.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). GMRES. ScholarGate. https://scholargate.app/numerical-methods/gmres