Conjugate Prior Analysis

A prior family F is conjugate to a likelihood p(x|θ) if for every prior in F, the posterior p(θ|x) is also in F. Formally, F is closed under Bayesian updating by that likelihood. The key consequence is that the posterior is characterised entirely by updated hyperparameters rather than an intractable unnormalised density.

Because hyperparameters correspond to pseudo-observations, prior elicitation is concrete: for a Beta prior, α and β represent imaginary prior successes and failures, so α + β is the total 'prior sample size.' Set them to reflect your genuine prior belief, or use small values (e.g. α = β = 1 for a uniform Beta) as a weakly informative choice. Always run a sensitivity analysis: vary the hyperparameters within a plausible range and confirm the posterior conclusion is stable.

Use conjugate analysis when your likelihood is in a conjugate family and the conjugate prior can adequately represent your beliefs. Switch to MCMC (or variational inference) when the likelihood is non-standard (e.g. logistic, survival, or hierarchical models), when the model has latent variables, or when richer prior families are needed. Conjugate results are also useful as quick sanity checks before running a full MCMC analysis.

Not in general. A uniform prior on one parameterisation implies a non-uniform (and possibly informative) prior after a reparameterisation. For conjugate models, the Jeffreys prior — derived from the Fisher information — is a principled approximately non-informative choice that is invariant to reparameterisation. For the Binomial likelihood it is Beta(0.5, 0.5), not the uniform Beta(1, 1).