Noninformative and Reference Priors

Noninformative and reference priors are constructed by formal rules so that the data, rather than the analyst's beliefs, dominate the posterior.

Definition

A noninformative prior is one chosen by a formal rule intended to exert minimal influence on the posterior; a reference prior is a particular objective prior defined to maximize the expected divergence between prior and posterior, thereby letting the data be maximally informative.

Scope

This topic covers the goals and constructions of objective priors: Laplace's flat prior, Jeffreys' invariant prior, Bernardo's reference priors based on maximizing expected information, the issue of improper priors and whether they yield proper posteriors, and known pathologies such as marginalization paradoxes.

Core questions

• What does it mean for a prior to be noninformative, and is true noninformativeness achievable?
• How is Jeffreys' prior derived and why is it invariant under reparameterization?
• How do Bernardo's reference priors formalize 'letting the data speak'?
• When do improper priors lead to improper or paradoxical posteriors?

Key concepts

• noninformative prior
• Jeffreys prior
• reference prior
• improper prior
• Fisher information
• invariance
• marginalization paradox

Key theories

Jeffreys' rule

Setting the prior proportional to the square root of the Fisher information determinant produces a prior invariant to smooth reparameterization, the canonical single-parameter objective prior.

Reference priors

Bernardo defined priors that maximize the expected Kullback-Leibler information the data provide about the parameter, with explicit treatment of nuisance parameters; these often differ from Jeffreys' prior in multiparameter problems.

Clinical relevance

Objective priors provide reproducible default analyses for regulatory and scientific reporting, where the appearance of injecting subjective belief is undesirable and a conventional baseline is needed.

History

Laplace used uniform priors under the principle of insufficient reason; Jeffreys introduced invariant priors in 1946; Bernardo formalized reference priors in 1979, later refined by Berger and Bernardo for multiparameter and ordered-nuisance settings.

Debates

Do truly noninformative priors exist?

Critics argue that no prior is genuinely uninformative because flatness is not invariant under reparameterization, while proponents of reference priors offer a principled, invariance-based construction.

Key figures

• Harold Jeffreys
• Jose-Miguel Bernardo
• Pierre-Simon Laplace
• James Berger

Seminal works

• jeffreys1946
• bernardo1979

Frequently asked questions

Is a uniform prior the same as a noninformative prior?

Not necessarily. A uniform prior is noninformative only on the chosen scale; after a nonlinear reparameterization it becomes informative, which is why invariance-based priors such as Jeffreys' are preferred for objective analysis.