Empirical Bayes

In full hierarchical Bayes, the hyperparameters η are assigned their own prior and integrated out along with θ, typically via MCMC. Empirical Bayes instead plugs in a point estimate η̂ obtained from the marginal likelihood, treating it as fixed. EB is faster and often gives very similar point estimates, but the resulting credible intervals are slightly too narrow because they ignore uncertainty in η̂. For large numbers of units the difference is usually small; for few units, full hierarchical Bayes is preferable.

The James-Stein estimator shrinks k simultaneous normal means toward a common value and provably has lower mean-squared error than the MLE when k ≥ 3. Efron and Morris (1973) showed that this estimator is the empirical Bayes posterior mean under a normal prior whose variance is estimated from the data, formally unifying the two ideas. The shrinkage factor B̂ plays the role of the estimated prior weight.

Exchangeability means that before seeing the data, all units' parameters are plausibly drawn from the same prior distribution. This fails when units are structurally different — for example, pooling gene expression estimates for housekeeping genes and regulatory genes under one prior, or pooling schools in rural and urban areas when their outcome distributions are known to differ systematically. In such cases, stratified or hierarchical models with group-specific priors are more appropriate.

Yes. The EB framework generalises to any likelihood within the exponential family. Poisson-gamma models are standard for count data (e.g. gene read counts in RNA-seq), beta-binomial models for proportion data, and gamma-gamma models for skewed positive outcomes. The marginal likelihood changes form accordingly, but the conceptual pipeline — estimate η̂ from the marginal, then compute the posterior — remains the same.