Conjugate Priors

A conjugate prior keeps the posterior in the same distributional family as the prior, turning Bayesian updating into a simple update of the family's parameters.

Definition

A family of priors is conjugate to a given likelihood if, for any data, the resulting posterior belongs to the same family; the posterior is obtained by updating the family's hyperparameters in closed form.

Scope

This topic covers the definition of conjugacy, the standard conjugate pairs (Beta-Binomial, Gamma-Poisson, Normal-Normal, Normal-inverse-Gamma, Dirichlet-Multinomial), the link to exponential families, and the interpretation of prior parameters as pseudo-counts or prior sample size.

Core questions

• What does it mean for a prior to be conjugate to a likelihood?
• Which conjugate pairs arise for the common exponential-family models?
• How do conjugate hyperparameters act as prior pseudo-data?
• Why does conjugacy follow from the structure of exponential families?

Key concepts

• conjugate prior
• Beta-Binomial
• Gamma-Poisson
• Normal-Normal
• Dirichlet-Multinomial
• exponential family
• hyperparameters
• prior pseudo-counts

Key theories

Exponential-family conjugacy

Diaconis and Ylvisaker characterized conjugate priors for exponential-family likelihoods and showed they imply posterior expectations that are linear in the sufficient statistics.

Prior as pseudo-data

Conjugate hyperparameters can be read as the counts and totals of an imaginary prior dataset, so the posterior combines real and prior pseudo-observations additively.

Clinical relevance

Conjugate models give fast, transparent updates that are widely used for proportion and rate estimation, adaptive randomization, and as building blocks inside larger sampling-based analyses.

History

Raiffa and Schlaifer systematized conjugate analysis for decision problems in 1961; Diaconis and Ylvisaker gave the general characterization for exponential families in 1979. Conjugacy remains central as a component within modern computational schemes such as Gibbs sampling.

Key figures

• Howard Raiffa
• Robert Schlaifer
• Persi Diaconis

Seminal works

• diaconis1979
• gelman2013

Frequently asked questions

Why use conjugate priors when computers can handle any prior?

Conjugate priors give exact closed-form posteriors that are fast and interpretable, and they often serve as the full-conditional updates inside Gibbs samplers even when the overall model is not conjugate.