Likelihood and Bayesian Updating
The likelihood carries all the information the data provide about the parameters, and Bayesian updating turns yesterday's posterior into today's prior as evidence accumulates.
Definition
The likelihood is the sampling density of the observed data viewed as a function of the parameters; Bayesian updating is the repeated application of Bayes' theorem so that information from successive observations is combined into a single posterior.
Scope
This topic covers the likelihood function and the likelihood principle, the sequential nature of Bayesian updating in which the posterior from one batch of data becomes the prior for the next, and the coherence of updating under exchangeable observations.
Core questions
- What is the likelihood function and why is it central to inference?
- What does the likelihood principle assert, and how does Bayesian inference satisfy it?
- How does the posterior from one dataset serve as the prior for the next?
- Why is sequential Bayesian updating order-invariant for exchangeable data?
Key concepts
- likelihood function
- likelihood principle
- sufficiency
- sequential updating
- prior-to-posterior recursion
- evidence accumulation
Key theories
- Likelihood principle
- Two experiments yielding proportional likelihood functions for the same parameter carry the same evidential information; Bayesian inference automatically respects this principle.
- Sequential updating
- Applying Bayes' theorem repeatedly is equivalent to applying it once to the pooled data, so beliefs can be revised online without storing the full dataset.
Clinical relevance
Sequential updating supports adaptive and interim analyses in clinical trials, online learning systems, and any setting where data arrive in a stream and beliefs must be revised continuously.
History
Fisher introduced the likelihood as a distinct concept in the 1920s; Birnbaum's 1962 analysis formalized the likelihood principle from sufficiency and conditionality. Bayesian theory absorbed these ideas, framing updating as iterated conditioning.
Debates
- Status of the likelihood principle
- Whether the likelihood principle should constrain all inference is contested, since many frequentist procedures (such as those using stopping rules) violate it while Bayesian methods do not.
Key figures
- Ronald A. Fisher
- Allan Birnbaum
- George Barnard
Related topics
Seminal works
- birnbaum1962
- robert2007
Frequently asked questions
- Does it matter in which order I update on different data points?
- For exchangeable observations the final posterior is the same regardless of the order in which the data are processed, because Bayesian updating is associative and equivalent to conditioning on all data at once.