Prior Elicitation and Sensitivity Analysis

Prior elicitation translates expert knowledge into probability distributions, and sensitivity analysis checks how much the conclusions depend on those prior choices.

Definition

Prior elicitation is the process of constructing a prior distribution from an expert's beliefs using structured judgments such as quantiles or probabilities; sensitivity (robust Bayesian) analysis quantifies how the posterior changes as the prior is varied within a plausible class.

Scope

This topic covers methods for eliciting subjective probabilities from experts, encoding them as prior distributions, and assessing robustness through sensitivity analysis over classes of priors, including the use of contamination classes and bounds on posterior quantities.

Core questions

• How are an expert's beliefs elicited and turned into a prior distribution?
• What biases affect probability judgments and how can elicitation mitigate them?
• How is robustness to the prior assessed across a class of distributions?
• When does prior choice materially change the conclusions of an analysis?

Key concepts

• prior elicitation
• expert judgment
• overconfidence bias
• robust Bayesian analysis
• contamination class
• sensitivity analysis

Key theories

Structured elicitation

Eliciting quantiles, probabilities, or comparisons and fitting a distribution to them produces reproducible priors while controlling well-documented judgment biases such as overconfidence.

Robust Bayesian analysis

Rather than a single prior, a class of priors is considered, and the range of resulting posterior quantities indicates whether conclusions are robust to prior specification.

Clinical relevance

Formal elicitation and sensitivity analysis are used to incorporate expert opinion in health technology assessment, environmental risk, and trial design, while demonstrating that conclusions are not artifacts of an arbitrary prior.

History

Structured elicitation protocols developed from decision analysis and psychology of judgment, consolidated in the 2006 SHELF-related literature. Robust Bayesian analysis, formalized by Berger and others from the 1980s, provided the complementary tools for assessing prior sensitivity.

Debates

How much should priors be allowed to drive conclusions?

Practitioners debate the acceptable degree of prior influence and how transparently sensitivity to the prior must be reported, especially in regulated decision-making.

Key figures

• Anthony O'Hagan
• James Berger
• Paul Garthwaite

Seminal works

• ohagan2006
• berger1990

Frequently asked questions

What should I do if my conclusions change a lot with the prior?

Strong sensitivity to the prior signals that the data are not very informative about the quantity of interest; the honest response is to report the dependence, collect more data, or justify the prior carefully rather than hide the sensitivity.