Normalizing Flows
Also known as: Flow-Based Generative Models, Invertible Neural Networks, Exact Likelihood Models, Akışa Dayalı Üretici Modeller
Normalizing flows are a class of generative models that learn a complex probability distribution by applying a sequence of invertible, differentiable transformations to a simple base distribution such as a standard Gaussian. Introduced by Rezende and Mohamed (2015) in the context of variational inference, they enable exact likelihood computation and efficient sampling, making them a principled alternative to VAEs and GANs for density estimation and generation tasks.
Key highlights
- Exact likelihood evaluation: unlike GANs or diffusion models, flows yield a tractable log-probability for any input.
- Efficient sampling: sampling requires only a single forward pass through the invertible network, with no iterative refinement.
- Invertibility enables both generation and inference: the same model performs density estimation and latent-space encoding without separate encoder/decoder networks.
- Flexible posterior approximation: when embedded in variational inference, flows produce richer, non-Gaussian posteriors that reduce the approximation gap.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use normalizing flows when you need exact and tractable likelihood evaluation alongside efficient sampling—tasks like density estimation, anomaly detection, variational inference with expressive posteriors, or probabilistic forecasting. They are best suited to continuous data with a moderate number of dimensions. Assume all transformations are differentiable and that Jacobian determinants can be computed efficiently. Normalizing flows become computationally costly in very high dimensions if the Jacobian is not structured (e.g., triangular). Consider diffusion models or score-based models when scalability to very high-dimensional image data is the primary concern.
Strengths & limitations
- Exact likelihood evaluation: unlike GANs or diffusion models, flows yield a tractable log-probability for any input.
- Efficient sampling: sampling requires only a single forward pass through the invertible network, with no iterative refinement.
- Invertibility enables both generation and inference: the same model performs density estimation and latent-space encoding without separate encoder/decoder networks.
- Flexible posterior approximation: when embedded in variational inference, flows produce richer, non-Gaussian posteriors that reduce the approximation gap.
- Architectural constraints: every layer must be invertible and have a tractable Jacobian determinant, which restricts the space of usable architectures.
- Dimensionality preservation: the latent space must have the same dimension as the data space, preventing the natural dimensionality reduction that VAEs can exploit.
- Scaling to high dimensions: computing or approximating Jacobian determinants becomes expensive for very high-dimensional data unless special structures (coupling layers, autoregressive layers) are used.
- Training stability: deep flow models with many stacked transformations can suffer from numerical instability and vanishing/exploding Jacobians.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How do normalizing flows differ from variational autoencoders?
VAEs use an approximate encoder and decoder, optimizing a lower bound on the likelihood; the posterior is typically assumed Gaussian, introducing an approximation gap. Normalizing flows compute the exact likelihood via invertible transformations and can be used to enrich the VAE posterior, eliminating the Gaussian assumption and tightening the bound without introducing a separate decoder.
Why must the transformation be invertible?
Invertibility guarantees that the change-of-variables formula applies exactly, allowing the model to compute the log-density of transformed samples by tracing back to the simple base distribution. Without invertibility, the Jacobian determinant is undefined or intractable, making exact likelihood computation impossible and breaking both training and density evaluation.
Can normalizing flows be used for discrete data such as text?
Not directly, because the change-of-variables formula requires continuous and differentiable mappings. Practitioners typically apply dequantization (adding continuous noise to discrete values) or work in a continuous embedding space. Dedicated discrete flow variants exist but involve additional approximations and are less mature than continuous flows.
Sources
- 1.Rezende, D. J., & Mohamed, S. (2015). Variational inference with normalizing flows. International Conference on Machine Learning (ICML), 1530–1538.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 2). Normalizing Flows. ScholarGate. https://scholargate.app/deep-learning/normalizing-flows