Robust Markov Chain Monte Carlo
Robust Markov Chain Monte Carlo Sampling · Also known as: robust MCMC, outlier-robust MCMC, robust posterior sampling, misspecification-robust MCMC
Robust MCMC combines Markov chain Monte Carlo sampling with robustness techniques to produce reliable posterior inference when data contain outliers, when the assumed model is misspecified, or when the target distribution has heavy tails that cause standard samplers to mix poorly or yield distorted estimates.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use robust MCMC when you suspect outliers or data contamination, when the assumed parametric model may be only approximately correct, or when pilot runs of standard MCMC show slow mixing, high R-hat values, or large leverage from individual observations. It is especially valuable in small-to-medium samples where a few unusual points can dominate. Avoid it when the data genuinely follow the assumed distribution without contamination — the robustification then adds computational cost and can reduce efficiency. Also avoid it when the goal is exact Bayesian inference under a specific model; robust modifications change the target distribution and should be motivated explicitly.
Strengths & limitations
- Posterior inference remains reliable even when a fraction of observations are outliers or come from a contaminating distribution.
- Handles heavy-tailed target distributions that cause standard Gaussian-proposal MCMC to mix slowly.
- Reduces sensitivity to model misspecification without abandoning the Bayesian probabilistic framework.
- Robust adaptation rules (median-based step size, MAD-based mass matrix) stabilise tuning in practice.
- Pairs naturally with robust model comparison criteria (WAIC, LOO-CV) to detect and quantify misspecification.
- Robustification modifies the target posterior; the resulting distribution may not correspond to any standard generative model, complicating interpretation.
- Choosing the robustness tuning parameter (e.g. beta in DPD, degrees of freedom in t-likelihood) requires additional calibration and sensitivity analysis.
- For fully clean, well-specified data, robust MCMC is less efficient than standard MCMC because heavy-tailed proposals or down-weighting waste information.
- Theoretical convergence guarantees are harder to establish than for standard MCMC, particularly for adaptive robust kernels.
Frequently asked
Is robust MCMC sampling the same as standard MCMC applied to a robust model?
They overlap but are not identical. Using a t-likelihood in a standard sampler is a model-level robustification. Robust MCMC also encompasses sampler-level modifications — heavy-tailed proposals, robust adaptation, kernel Stein discrepancy targets — that harden the algorithm itself, independently of whether the likelihood has been changed.
How do I choose the robustness tuning parameter?
Cross-validation (LOO-CV or k-fold) on predictive performance is the most common guide: fit the model at a grid of robustness values and pick the one with the best held-out log score. A sensitivity analysis checking how key posterior summaries shift across the grid is also essential.
Will robust MCMC always converge to the correct posterior?
Standard MCMC convergence theory guarantees apply when the robust target density is proper and the kernel satisfies the usual irreducibility and aperiodicity conditions. However, if the robust target is defined implicitly (e.g. via a kernel discrepancy objective), convergence proofs may require additional conditions; always verify with R-hat and effective sample size diagnostics.
When should I prefer robust MCMC over outlier removal followed by standard MCMC?
Prefer robust MCMC when outlier membership is uncertain or continuous (each observation has a degree of anomaly) rather than a binary label. Hard removal discards information and can introduce selection bias; down-weighting retains all data while limiting the influence of the most discordant points.
Does robust MCMC change model interpretation?
Yes. The posterior under a robust likelihood is not the exact Bayesian posterior under the stated model; it corresponds to a tilted or down-weighted version. Coefficient estimates and credible intervals should be interpreted relative to the robust target, and this should be reported explicitly in any analysis.
Sources
- Roberts, G. O. & Rosenthal, J. S. (2004). General state space Markov chains and MCMC algorithms. Probability Surveys, 1, 20–71. DOI: 10.1214/154957804100000024 ↗
- Barp, A., Kennedy, C., Durmus, A. & Girolami, M. (2022). Targeted separation and convergence with kernel discrepancies. arXiv preprint. link ↗
How to cite this page
ScholarGate. (2026, June 3). Robust Markov Chain Monte Carlo Sampling. ScholarGate. https://scholargate.app/en/bayesian/robust-markov-chain-monte-carlo
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Gibbs SamplingBayesian↔ compare
- Hamiltonian Monte CarloBayesian↔ compare
- MCMCBayesian↔ compare
- Robust Bayesian InferenceBayesian↔ compare
- Sequential Monte CarloBayesian↔ compare