Non-negative Matrix Factorization (NMF)
Non-negative Matrix Factorization (Lee & Seung, 1999) · Also known as: NMF, NNMF, nonnegative matrix factorization, non-negative matrix approximation, parts-based matrix decomposition
Non-negative Matrix Factorization (NMF) is a family of algorithms, introduced by Lee and Seung in their landmark 1999 Nature paper, that decomposes a non-negative data matrix V into the product of two lower-rank non-negative matrices W (basis components) and H (encoding coefficients). Unlike PCA or SVD, the non-negativity constraint forces the algorithm to learn strictly additive, parts-based representations, making the factors directly interpretable as building blocks of the original data.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+1 more
When to use it
NMF is appropriate when the data are non-negative by nature — pixel intensities, word counts, spectral amplitudes, gene expression levels — and when interpretability of the latent components matters. It is well-suited to topic modelling of text corpora, hyperspectral image unmixing, audio source separation, and bioinformatics applications such as identifying gene expression signatures. A key assumption is that the observed data are plausibly generated as non-negative combinations of underlying non-negative patterns; if the data contain negative values, NMF is not applicable without preprocessing. The number of components k must be specified in advance, so model selection (e.g., via reconstruction error, stability across runs, or domain knowledge) is part of the workflow.
Strengths & limitations
- Non-negativity produces parts-based, additive representations that are directly interpretable without sign conventions.
- Multiplicative update rules are simple to implement, preserve non-negativity automatically, and have guaranteed monotone convergence.
- Competitive with or superior to PCA and VQ for learning parts of objects such as facial features and semantic topics in text.
- Well-established theoretical and algorithmic foundations, with decades of extensions covering sparsity, robustness, and probabilistic formulations.
- Applicable to a wide range of domains wherever data are inherently non-negative: images, text, audio, spectral data, and genomics.
- Non-convex optimisation means that results depend on initialisation and convergence to a global minimum is not guaranteed.
- The number of components k must be chosen prior to fitting; there is no automatic selection and model comparison can be computationally expensive.
- Not applicable to data containing negative values without domain-specific preprocessing that may distort the analysis.
- Convergence of multiplicative update rules can be slow for large matrices; projected gradient and alternating least squares variants are often faster in practice.
- Uniqueness of the factorisation is generally not guaranteed unless additional constraints such as sparsity or orthogonality are imposed.
Frequently asked
How is NMF different from PCA or SVD?
PCA and SVD impose orthogonality but allow both positive and negative values in the components, which can produce holistic, hard-to-interpret representations (e.g., a PCA face component mixes positive and negative pixel weights). NMF forbids negative values entirely, so every component is a genuine part that adds to, never subtracts from, the reconstruction. The trade-off is that NMF components are not orthogonal and the solution is not unique without extra constraints.
How do I choose the number of components k?
There is no universally optimal criterion. Common approaches include plotting reconstruction error against k and looking for an elbow, assessing the stability of components across multiple random initialisations (cophenetic correlation coefficient), and using cross-validation on held-out entries of V. Domain knowledge — for instance, the expected number of topics in a corpus or spectral endmembers in an image — should also inform the choice.
Why do I get different results each time I run NMF?
NMF optimises a non-convex objective, so different random initialisations can converge to different local minima. The standard remedy is to run NMF multiple times (e.g., 10–50 runs) with different random seeds and select the run with the lowest final reconstruction error. The NNDSVD initialisation provides a deterministic starting point that often reduces sensitivity to random seed.
Can NMF be used for topic modelling instead of LDA?
Yes. NMF on a TF-IDF or raw count document-term matrix produces topics (basis vectors of words) and document-topic loadings (encoding matrix) that are qualitatively similar to those from Latent Dirichlet Allocation. NMF is simpler to implement and faster to fit, while LDA offers a generative probabilistic model with interpretable Dirichlet priors. In practice both methods are competitive, and the choice often depends on whether probabilistic uncertainty estimates are needed.
Sources
- Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755), 788–791. DOI: 10.1038/44565 ↗
- Lee, D. D., & Seung, H. S. (2001). Algorithms for non-negative matrix factorization. Advances in Neural Information Processing Systems, 13, 556–562. link ↗
- Cichocki, A., Zdunek, R., Phan, A. H., & Amari, S. (2009). Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley. ISBN: 978-0-470-74666-0
How to cite this page
ScholarGate. (2026, June 3). Non-negative Matrix Factorization (Lee & Seung, 1999). ScholarGate. https://scholargate.app/en/machine-learning/non-negative-matrix-factorization
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Independent Component AnalysisMachine learning↔ compare
- K-Means ClusteringMachine learning↔ compare
- Latent Dirichlet AllocationMachine learning↔ compare
- Singular Value DecompositionNumerical Methods↔ compare