Bayesian Confirmatory Research — Bayesian Hypothesis Testing
Bayesian Confirmatory Research Design · Also known as: Bayesian hypothesis testing, confirmatory Bayesian analysis, Bayes factor hypothesis testing, BCR
Bayesian confirmatory research is a quantitative framework that tests pre-specified hypotheses by computing the Bayes factor — a ratio expressing how much more likely the observed data are under one hypothesis than another. Unlike classical null-hypothesis significance testing (NHST), it provides direct evidence for both the alternative and the null hypothesis, supports optional stopping rules under certain conditions, and updates prior beliefs with observed data through Bayes' theorem.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Bayesian confirmatory research is appropriate when the goal is to test a pre-specified directional or point hypothesis and when evidence for the null hypothesis is as informative as evidence against it — for example, when confirming that an effect is negligible or replicating a prior finding. It suits experimental, quasi-experimental, and survey designs where sample sizes are moderate to large and reasonable prior information about effect size exists. It is particularly valuable in replication studies, pre-registered clinical trials, and cumulative science programs. Avoid it when no defensible prior can be specified (pure exploration), when the statistical model is too complex for tractable Bayes factor computation, or when stakeholders require strict Type I error control under a Neyman-Pearson decision framework.
Strengths & limitations
- Provides direct, interpretable evidence for both the alternative and the null hypothesis — unlike p-values, which cannot support H0.
- Bayes factors accumulate continuously, enabling principled sequential testing and early stopping without inflating error rates.
- Pre-registration of hypotheses and priors makes the confirmatory claim transparent and reproducible.
- Effect-size posteriors communicate uncertainty about the magnitude of the effect, not just binary reject/retain decisions.
- Naturally integrates prior knowledge from meta-analyses or theory, making cumulative science explicit.
- The Bayes factor is sensitive to the choice of prior: poorly specified or overly diffuse priors can yield misleading evidence, especially in small samples.
- Computing Bayes factors for complex models (e.g., multilevel, structural equation) requires custom numerical methods and is not yet routine.
- Confirmatory validity depends entirely on genuine pre-registration; post-hoc Bayesian analysis is exploratory, not confirmatory, regardless of labeling.
- Interpreting and reporting Bayes factors remains unfamiliar to many reviewers and editors trained solely in NHST, which can complicate peer review.
Frequently asked
What is the difference between Bayesian confirmatory research and Bayesian exploratory analysis?
Confirmatory Bayesian research requires that hypotheses, priors, and the analysis plan are pre-registered before data collection. The resulting Bayes factor is a confirmatory test of a pre-specified claim. Bayesian exploratory analysis applies Bayesian methods to unplanned or data-driven questions; the computations are valid, but the epistemic status is exploratory — findings require independent replication before being treated as confirmed.
Can Bayesian confirmatory research replace NHST entirely?
For many confirmatory applications, Bayes factors provide more informative evidence than p-values. However, NHST with strict alpha control remains appropriate in regulatory contexts (e.g., FDA drug approval) where Type I error must be bounded by a formal guarantee. The two frameworks answer subtly different questions, and some researchers use both in a complementary reporting style.
How do I choose the prior distribution?
The starting point is whether informative prior knowledge exists. If a recent meta-analysis provides a reliable estimate of the effect size distribution, use it to construct an informed prior. If not, the default Cauchy prior (scale r = 0.707 for medium effects) is a well-defended generic choice for two-group comparisons. Always report a sensitivity analysis showing how the BF changes across a plausible range of prior scales.
What software can I use?
JASP (jasp-stats.org) offers a free, point-and-click interface for Bayes factors in common designs (t-tests, ANOVA, regression, contingency tables). The BayesFactor package in R provides the same computations with scripting flexibility. For custom or hierarchical models, Stan (mc-stan.org) with bridge sampling via the bridgesampling R package computes marginal likelihoods and thus Bayes factors.
What sample size do I need?
Bayesian design analysis (Schoenbrodt & Wagenmakers, 2018) replaces classical power analysis: you simulate studies across a range of plausible effect sizes and determine the sample size (or maximum N for sequential designs) needed to achieve a target BF (e.g., BF > 10 with 90% probability). The BayesFactor R package and JASP include tools for this calculation.
Sources
- Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. DOI: 10.3758/PBR.16.2.225 ↗
- Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, A. J., Love, J., Selker, R., Gronau, Q. F., Smira, M., Epskamp, S., Matzke, D., Rouder, J. N., & Morey, R. D. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin & Review, 25(1), 35–57. DOI: 10.3758/s13423-017-1343-3 ↗
How to cite this page
ScholarGate. (2026, June 3). Bayesian Confirmatory Research Design. ScholarGate. https://scholargate.app/en/research-design/bayesian-confirmatory-research
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Power analysisStatistics↔ compare