Jensen-Shannon Divergence
Jensen-Shannon Information Divergence · Also known as: JS divergence, symmetric KL divergence, JS distance
Jensen-Shannon divergence is a symmetric information-theoretic measure of the difference between two probability distributions. Developed by Jian Lin in 1991 as a refinement to the asymmetric Kullback-Leibler divergence, it overcomes KL's directional limitation by averaging the divergences in both directions. The result is a true metric (satisfying triangle inequality) that ranges from 0 (identical distributions) to 1, making it suitable for symmetric comparison tasks.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Jensen-Shannon divergence is ideal when comparing probability distributions symmetrically without a reference or preference direction. Use it in clustering distributions, assessing model similarity, comparing language corpora, and decision-making where both alternatives are equally valid. It is more robust than KL divergence because it is symmetric and handles zero probabilities more gracefully. Suitable for any application requiring a true metric on probability distributions.
Strengths & limitations
- Symmetric: JS(P||Q) = JS(Q||P), treating both distributions equally
- True metric: satisfies non-negativity, identity, symmetry, and triangle inequality
- Finite for all probability distributions; avoids infinite divergence
- More robust to zero probabilities than KL divergence
- Computationally more expensive than KL divergence (requires two KL evaluations)
- Square root of JS divergence gives a metric, but JS itself does not directly quantify information loss
- Less interpretable than KL divergence in information-theoretic terms
- Slower convergence properties in optimization algorithms compared to KL divergence
Frequently asked
How does Jensen-Shannon differ from Kullback-Leibler divergence?
KL divergence is asymmetric and directional; Jensen-Shannon is symmetric. KL(P||Q) can be infinite, while JS is always finite. Jensen-Shannon is easier to use when comparing distributions without a clear reference direction, but KL is more interpretable from an information-theoretic perspective.
Is Jensen-Shannon divergence a true metric?
Jensen-Shannon divergence itself is a divergence (non-negative, zero iff P=Q, symmetric), but does not satisfy the triangle inequality. However, the square root of Jensen-Shannon divergence is a true metric in the mathematical sense.
Can I use Jensen-Shannon with non-probability vectors?
No. Jensen-Shannon requires valid probability distributions (non-negative values summing to 1). Normalize your data first (e.g., divide by the sum) to create a probability distribution.
How does Jensen-Shannon handle zero probabilities?
Jensen-Shannon is more robust than KL divergence with zero probabilities. By averaging distributions before computing divergence, it reduces the risk of undefined values. However, proper normalization and handling of edge cases is still recommended.
Sources
- Lin, J. (1991). Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory, 37(1), 145-151. DOI: 10.1109/18.61115 ↗
- Cover, T. M., & Thomas, J. A. (1991). Elements of Information Theory. Wiley-Interscience. DOI: 10.1002/0471200611 ↗
How to cite this page
ScholarGate. (2026, June 3). Jensen-Shannon Information Divergence. ScholarGate. https://scholargate.app/en/decision-making/jensen-shannon-divergence
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Hellinger DistanceDecision-making↔ compare
- Kullback-Leibler DivergenceDecision-making↔ compare