Prior Distributions
The prior distribution encodes what is known about the parameters before the data are seen, and its specification is the distinctive modeling step of Bayesian analysis.
Definition
A prior distribution is a probability distribution over the unknown parameters of a model that represents information or assumptions available before observing the current data, and that is combined with the likelihood to form the posterior.
Scope
This area covers the families and principles used to construct priors: conjugate families chosen for analytical convenience, noninformative and reference priors designed to minimize influence, weakly informative priors used for regularization, and the elicitation and sensitivity analysis that govern responsible prior choice.
Sub-topics
Core questions
- What makes a prior conjugate, and why is conjugacy useful?
- How are noninformative or reference priors constructed and justified?
- When are weakly informative priors preferable to flat priors?
- How is prior information elicited and how is sensitivity to the prior assessed?
Key concepts
- prior distribution
- conjugate prior
- noninformative prior
- reference prior
- Jeffreys prior
- weakly informative prior
- improper prior
- prior sensitivity
Key theories
- Conjugacy
- A prior is conjugate to a likelihood when the posterior stays in the same family, giving closed-form updating; conjugate priors arise naturally for exponential-family likelihoods.
- Jeffreys' invariant prior
- Jeffreys' rule sets the prior proportional to the square root of the Fisher information determinant, yielding a prior invariant under reparameterization and a canonical objective default.
- Weakly informative priors
- Priors that are deliberately broad but proper provide regularization and computational stability without imposing strong substantive beliefs, an approach emphasized in modern applied Bayesian work.
Clinical relevance
Prior choice determines how much external evidence enters an analysis, which is consequential in small-sample settings such as early-phase trials, rare-disease genetics, and risk assessment, where well-chosen priors stabilize estimates.
History
Laplace's principle of insufficient reason offered the first default prior. Jeffreys formalized invariant objective priors in the 1940s; Bernardo introduced reference priors in 1979; and the modern applied tradition has favored weakly informative priors for both regularization and computational reliability.
Debates
- Flat versus weakly informative priors
- Whether 'noninformative' flat priors are genuinely neutral is disputed, since they can be improper or imply strong beliefs on transformed scales, motivating weakly informative alternatives.
Key figures
- Harold Jeffreys
- Jose-Miguel Bernardo
- Edwin T. Jaynes
- Andrew Gelman
Related topics
Seminal works
- gelman2013
- jeffreys1946
Frequently asked questions
- Can I just use a flat prior to be objective?
- A flat prior is not automatically neutral: it may be improper, may not yield a proper posterior, and can be highly informative after a change of variables, so weakly informative proper priors are often preferred.
- Does the prior stop mattering with enough data?
- Under regularity conditions the likelihood dominates as the sample grows and the posterior becomes insensitive to a reasonable prior, but with small samples or many parameters the prior can remain influential.