Importance Sampling — Variance Reduction for Rare Events
Importance Sampling (Variance Reduction Monte Carlo) · Also known as: IS, weighted Monte Carlo, Önem Örneklemesi
Importance sampling is a Monte Carlo variance-reduction technique that shifts the sampling distribution toward the region of interest — typically a rare or extreme event — so that informative samples are drawn far more often than under the original distribution. Developed at the RAND Corporation by Herman Kahn and Theodore Harris around 1951, it makes tail-probability estimation (such as Value-at-Risk or system-failure probability) tractable where standard Monte Carlo would require an astronomically large number of runs.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Importance sampling is appropriate when the probability or expectation to be estimated involves a rare event that standard Monte Carlo cannot reach in a reasonable number of runs — typically tail probabilities below about 10⁻². Classic use cases include Value-at-Risk and Expected Shortfall in finance, system-failure probability in reliability engineering, and particle transport in nuclear physics. The method requires that you can evaluate the likelihood ratio p(x)/q(x) pointwise and that this ratio has finite variance under q; if the proposal is poorly matched to the target, weight variance explodes and the method performs worse than crude Monte Carlo. Minimum meaningful run counts depend on the rarity of the event but are often in the thousands to tens of thousands.
Strengths & limitations
- Reduces estimator variance by orders of magnitude for rare-event probabilities compared with crude Monte Carlo.
- Produces an unbiased estimate regardless of the choice of proposal, provided the likelihood ratio is computable.
- Flexible: the proposal distribution can be adapted iteratively (cross-entropy method, sequential IS) as more information about the target region becomes available.
- Works with continuous, mixed, or even discrete distributions as long as the likelihood ratio is well defined.
- Selecting a good proposal distribution requires problem-specific insight; a poor proposal can increase rather than decrease variance.
- The likelihood-ratio weights must have finite variance under q; heavy-tailed or mismatched proposals can cause weight degeneracy.
- Performance degrades in high-dimensional spaces because the optimal proposal becomes harder to approximate — curse of dimensionality.
- Diagnosing proposal quality requires monitoring auxiliary quantities (ESS, weight CV) that add complexity beyond a naive simulation.
Frequently asked
How is importance sampling different from standard Monte Carlo?
Standard (crude) Monte Carlo draws samples from the original distribution p and counts how often the rare event occurs; when the event is very unlikely, almost no samples hit the region and variance is enormous. Importance sampling draws from a different proposal distribution q that covers the rare region more densely, then corrects each sample by the likelihood ratio p(x)/q(x) to maintain an unbiased estimate. The result is the same expectation but with far lower variance when q is well chosen.
How do I choose the proposal distribution?
The theoretically optimal proposal is proportional to |f(x)| p(x), but this is rarely available in closed form. Practical strategies include exponential tilting (shifting the mean of a normal or exponential distribution toward the tail), the cross-entropy method (iteratively minimising KL divergence between q and the optimal proposal), or problem-specific constructions based on domain knowledge. After any choice, always compute the ESS to verify that the proposal is adequate.
What is the effective sample size (ESS) and why does it matter?
ESS = (Σwᵢ)² / Σwᵢ² summarises how many independent, equal-weight samples the weighted sample is worth. An ESS close to N indicates a well-matched proposal; an ESS much smaller than N means a few extreme weights dominate, inflating apparent precision while hiding large uncertainty. As a rule of thumb, an ESS below 10–20% of N warrants revising the proposal before trusting the estimates.
Can importance sampling be used in high dimensions?
With difficulty. In high dimensions the likelihood-ratio weights become highly variable (weight degeneracy), which collapses the ESS. Remedies include sequential importance sampling / particle filters, conditioning on low-dimensional sufficient statistics, or splitting the rare event into a sequence of less-rare events (multilevel splitting). For moderate dimensions (up to roughly 10–20) with a well-designed proposal the method remains practical.
Sources
- Rubinstein, R.Y. & Kroese, D.P. (2016). Simulation and the Monte Carlo Method (3rd ed.). Wiley. DOI: 10.1002/9781118631980 ↗
- Glasserman, P. (2003). Monte Carlo Methods in Financial Engineering. Springer. DOI: 10.1007/978-0-387-21617-1 ↗
How to cite this page
ScholarGate. (2026, June 1). Importance Sampling (Variance Reduction Monte Carlo). ScholarGate. https://scholargate.app/en/simulation/importance-sampling
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Extreme Value TheoryFinance↔ compare
- Latin Hypercube SamplingSimulation↔ compare
- MONTE-CARLO-SIMULATIONDecision-making↔ compare
- Stratified SamplingSurvey Methodology↔ compare
- Value at RiskFinance↔ compare