Singular Value Decomposition
Also known as: SVD, thin SVD, reduced SVD
Singular Value Decomposition (SVD) is a fundamental matrix factorization technique that decomposes any m × n matrix A into the product A = U Σ V^T, where U and V are orthogonal matrices and Σ is a diagonal matrix of singular values. Developed by Gene Golub and others in the 1960s–1970s, SVD is the most robust method for analyzing matrix structure and solving linear systems.
Key highlights
- Identifies and separates signal from noise; truncated SVD retains dominant components while suppressing small singular values
- Reveals matrix rank exactly (number of non-negligible singular values); determines numerical rank via threshold
- Orthogonal decomposition provides stable basis for projections and pseudo-inverse computation
- Applicable to rectangular, singular, and complex matrices; no assumptions on positive-definiteness
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use SVD for solving ill-posed or rank-deficient systems, computing matrix rank and null space, image compression, noise filtering, and dimensionality reduction. Essential when matrix conditioning is poor or rank is unknown. For large sparse matrices, iterative methods (Lanczos, LOBPCG) are preferable; full SVD costs O(m n²) or O(m² n).
Strengths & limitations
- Identifies and separates signal from noise; truncated SVD retains dominant components while suppressing small singular values
- Reveals matrix rank exactly (number of non-negligible singular values); determines numerical rank via threshold
- Orthogonal decomposition provides stable basis for projections and pseudo-inverse computation
- Applicable to rectangular, singular, and complex matrices; no assumptions on positive-definiteness
- Computational cost is O(m²n) or O(mn²) for full SVD; prohibitive for very large matrices unless sparse structure is exploited
- Full SVD stores O(mn) memory, limiting use on high-dimensional data; thin SVD or sketching methods reduce cost
- For large sparse matrices, full SVD destroys sparsity; iterative partial SVD (Lanczos) is necessary
- Singular vectors are not unique if singular values repeat; truncated SVD introduces ambiguity in principal subspace
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Why is SVD more stable than LU factorization for solving Ax = b?
LU can have large factors (growth factor > 1), amplifying roundoff errors for ill-conditioned A. SVD's orthogonal factors (U, V) have perfect condition number 1; regularizing via truncated SVD is straightforward. For ill-posed problems, this difference is critical.
How do I determine the numerical rank of a matrix?
Compute SVD and count singular values above a threshold (e.g., σ_i > ε ||σ_1||, where ε ≈ 10^{-14} machine precision times m or n). Alternatively, use the L-curve criterion or generalized cross-validation for data-dependent thresholding.
What is the difference between full and thin SVD?
Full SVD returns U (m × m) and V (n × n). Thin SVD returns U (m × r) and V (n × r), where r = rank(A) ≤ min(m,n). Thin SVD is more memory-efficient and sufficient for most applications; use full SVD only when you need a complete orthogonal basis for the null space.
Can I use SVD for eigenvalues of a square matrix?
Yes: if A is square and symmetric, the SVD singular values are |λ_i| (absolute eigenvalues) and U = V. For non-symmetric A, use eigendecomposition directly (QR iteration, Krylov methods) instead—SVD is overkill and doesn't recover the eigenvectors.
Sources
- 1.Golub, G. H., & Kahan, W. (1970). Calculating the singular values and pseudo-inverse of a matrix. Journal of the SIAM Series B: Numerical Analysis, 2(2), 205–224.
- 2.Golub, G. H., & Van Loan, C. F. (1983). Matrix computations (2nd ed.). Johns Hopkins University Press.ISBN 0801854148
- 3.Trefethen, L. N., & Bau, D. (1997). Numerical Linear Algebra. SIAM.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Singular Value Decomposition. ScholarGate. https://scholargate.app/numerical-methods/singular-value-decomposition