Hellinger Distance
Also known as: Bhattacharyya distance, Hellinger metric
Hellinger distance is a symmetric, bounded metric that measures the difference between two probability distributions. Rooted in the work of Ernst Hellinger (1909) and later formalized in statistical divergence by Anil Bhattacharyya (1946), this distance ranges from 0 (identical distributions) to 1. It is a true metric satisfying all mathematical distance properties and is particularly well-suited for comparing probability distributions in a symmetric, numerically stable manner.
Key highlights
- True metric: satisfies non-negativity, identity, symmetry, and triangle inequality
- Symmetric: H(P||Q) = H(Q||P), treating both distributions equally
- Bounded: ranges from 0 to 1, making values easy to interpret and compare
- Numerically stable: handles zero and small probabilities without special cases
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Hellinger distance is ideal for comparing probability distributions when you need a symmetric, numerically stable metric. Use it in statistical hypothesis testing, distribution clustering, and decision-making with probabilistic profiles. It performs well with small or zero probabilities and is robust in high-dimensional spaces. Suitable for any application where a true metric on probability distributions is required without the directionality constraints of KL divergence.
Strengths & limitations
- True metric: satisfies non-negativity, identity, symmetry, and triangle inequality
- Symmetric: H(P||Q) = H(Q||P), treating both distributions equally
- Bounded: ranges from 0 to 1, making values easy to interpret and compare
- Numerically stable: handles zero and small probabilities without special cases
- Computationally more involved than simpler metrics due to square root operations
- Not directly interpretable in information-theoretic terms like KL divergence
- Less commonly used in optimization algorithms compared to KL divergence
- May emphasize small differences when probabilities are very small
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does Hellinger distance compare to Kullback-Leibler divergence?
Hellinger distance is symmetric, bounded [0,1], and a true metric. KL divergence is asymmetric, unbounded, and not a metric. Hellinger is more suitable when you want a symmetric comparison; KL is better for information-theoretic interpretation and optimization.
Is Hellinger distance appropriate for discrete and continuous distributions?
Hellinger distance works for discrete distributions directly. For continuous distributions, replace sums with integrals. In practice, continuous distributions are discretized or approximated (e.g., as mixtures), then the discrete Hellinger distance is applied.
Why is Hellinger distance bounded between 0 and 1?
The square root transformation and the specific normalization (division by sqrt(2)) ensure the distance ranges from 0 to 1. This bound emerges naturally from the Cauchy-Schwarz inequality applied to the probability square roots.
How does Hellinger distance handle probability distributions with zeros?
Hellinger distance handles zero probabilities naturally. The square root of zero is zero, and the distance remains well-defined. Unlike KL divergence, there is no singularity issue, making Hellinger more numerically stable.
Sources
- 1.Hellinger, E. (1909). Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. Journal für die Reine und Angewandte Mathematik, 136, 210-271.
- 2.Bhattacharyya, A. (1946). On a measure of divergence between two multinomial populations. Sankhya, 7, 401-406.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Hellinger Distance. ScholarGate. https://scholargate.app/decision-making/hellinger-distance