GMRES
Generalized Minimal Residual Method · Also known as: GMRES(m), restarted GMRES, Krylov-GMRES
GMRES (Generalized Minimal Residual) is an iterative method for solving large sparse non-symmetric or nonsymmetric linear systems Ax = b, developed by Saad and Schultz in 1986. It builds an orthonormal Krylov basis using Arnoldi's method and solves a least-squares problem to minimize residual at each iteration.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use GMRES for non-symmetric or ill-conditioned systems where CG is not applicable. Essential for advection-dominated PDEs, coupled multiphysics problems, and indefinite matrices. Restart parameter m = 30–40 balances memory and convergence. For symmetric matrices, CG or MINRES are more efficient.
Strengths & limitations
- Handles arbitrary non-symmetric and complex matrices; makes no positivity assumptions
- Monotonic reduction in residual norm guarantees progress; residual history is reliable
- Flexible preconditioning available (FGMRES); works with left or right preconditioners
- Proven convergence: eventually (possibly after many iterations) reaches any prescribed tolerance
- Full GMRES requires storing m Krylov vectors; memory demand scales as O(m·n), making it expensive for large m
- Cost per iteration grows as O(k·n) for iteration k; restarting every m steps mitigates but incurs convergence penalty
- Without preconditioning, convergence is often slow for ill-conditioned problems
- Householder or Givens orthogonalization required for stability; adds computational overhead
Frequently asked
What is the Krylov subspace and why does Arnoldi orthogonalization matter?
The Krylov subspace K_k(A, r₀) = span{r₀, A r₀, A² r₀, …, A^{k-1} r₀} contains increasingly rich information about A. Arnoldi orthogonalization produces an orthonormal basis for this space, enabling stable least-squares minimization of residuals.
Why restart at all if full GMRES converges?
Full GMRES stores all k Krylov vectors, demanding O(k·n) memory and O(k²) operations per iteration. After m = 30–50 iterations, memory exceeds availability. Restarting discards old vectors and restarts the process, trading some convergence speed for feasibility on large systems.
What restart parameter m should I choose?
m = 30 is a practical default; increase to 50–100 if memory allows and convergence is slow, decrease to 10–20 if memory is tight. Problem-dependent: elliptic PDEs often converge in 10–20 iterations; advection-dominated or coupled systems may need 40–100.
How does left preconditioning differ from right preconditioning in GMRES?
Left preconditioning solves M⁻¹ A x = M⁻¹ b, improving conditioning of M⁻¹ A. Right preconditioning solves A M⁻¹ y = b with x = M⁻¹ y, leaving the spectrum of A M⁻¹ unchanged. Left improves convergence rate; right avoids applying M⁻¹ at the end. FGMRES allows flexible (variable) preconditioners in right positioning.
Sources
- Saad, Y., & Schultz, M. H. (1986). GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM Journal on Scientific and Statistical Computing, 7(3), 856–869. DOI: 10.1137/0907058 ↗
- Walker, H. F. (1988). Implementation of the GMRES method using Householder reflections. SIAM Journal on Scientific and Statistical Computing, 9(1), 152–163. DOI: 10.1137/0909010 ↗
- Saad, Y. (2003). Iterative Methods for Sparse Linear Systems (2nd ed.). SIAM. DOI: 10.1137/1.9780898718003 ↗
How to cite this page
ScholarGate. (2026, June 3). Generalized Minimal Residual Method. ScholarGate. https://scholargate.app/en/numerical-methods/gmres
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Conjugate Gradient MethodNumerical Methods↔ compare