Simulation-Assisted Confirmatory Research
Also known as: simulation-based confirmatory design, Monte Carlo confirmatory research, computational confirmatory study, simulation-enhanced hypothesis testing
Simulation-assisted confirmatory research integrates computational simulation — most commonly Monte Carlo methods — into a hypothesis-driven, confirmatory study design. Before or alongside empirical data collection, the researcher runs simulated data under specified model assumptions to establish expected parameter distributions, verify statistical power, and anticipate the behavior of the chosen analysis. The empirical findings are then evaluated against those simulation-derived benchmarks, strengthening the evidential value of confirmatory conclusions.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Simulation-assisted confirmatory research is appropriate when a research question demands hypothesis testing with rigorous control of Type I and Type II error rates, and when the analyst cannot rely on closed-form power formulas because the design is complex (e.g., multilevel models, latent variable structures, non-normal outcomes). It is especially valuable for pre-registering studies where stakeholders require transparent justification of sample size and analysis choices. It is not appropriate for purely exploratory work where hypotheses are still being formed, for very simple designs where analytic power formulas already exist and suffice, or when the researcher lacks the computational skills or software infrastructure to implement credible simulation.
Strengths & limitations
- Produces accurate, design-specific power estimates for complex models where textbook formulas do not apply.
- Exposes potential estimation problems (non-convergence, boundary solutions, inflated Type I error) before expensive data collection begins.
- Strengthens the credibility of confirmatory conclusions by grounding them in a pre-specified, simulation-validated analysis plan.
- Facilitates transparent pre-registration by providing concrete, quantitative justification for sample size and analytic choices.
- Can accommodate realistic violations of assumptions (non-normality, missing data, measurement error) that textbook formulas ignore.
- Simulation results are only as valid as the assumed data-generating model; if the real population differs substantially from the simulated one, power and error-rate estimates may be misleading.
- Requires programming competence (R, Python, or specialist software) and considerable time investment before any empirical data are collected.
- The combinatorial space of simulation parameters can be large, and researchers may inadvertently simulate a favorable scenario rather than the realistic one.
- Does not substitute for substantive theoretical justification of the hypothesized effect size; a simulation run on an implausibly large assumed effect produces optimistic but misleading power estimates.
Frequently asked
How is this different from a simulation study on its own?
A standalone simulation study uses synthetic data to evaluate the behavior of a statistical method under controlled conditions and is itself the research product. Simulation-assisted confirmatory research uses simulation as a planning and benchmarking tool in service of a hypothesis test conducted on real empirical data. The simulation informs the design and provides a reference frame, but the confirmatory conclusion rests on actual data.
What software is typically used for the simulation component?
R is the most widely used environment, with packages such as simstudy, simr (for mixed models), lavaan and simsem (for SEM), and the general-purpose furrr or future for parallelization. Python with NumPy and SciPy is also common. Specialist tools such as G*Power handle some scenarios analytically, but for complex designs custom Monte Carlo scripts are generally required.
How many simulation iterations are enough?
1,000 iterations are often cited as a minimum for stable power estimates; 5,000–10,000 are preferred when estimating Type I error rates or when model convergence rates need precise characterization. The precision of simulation-based power estimates depends on the number of iterations: more iterations narrow the Monte Carlo standard error around the estimated power.
Does running a simulation before the study count as pre-registration?
Running a simulation is not itself pre-registration, but it produces the quantitative evidence that makes a pre-registration meaningful. The pre-registration document should include the simulation code, the assumed parameter values and their justification, the resulting power estimate, and the exact analysis pipeline to be applied to the empirical data. Uploading all of these to a registry (OSF, AsPredicted) before data collection is what constitutes pre-registration.
What if my empirical results deviate substantially from simulation expectations?
Deviation is informative, not a failure. It should prompt investigation of whether the real population differed from the assumed model (e.g., lower true effect size, greater variance), whether there were data-collection issues, or whether the analysis encountered numerical problems that the simulation flagged as possible. Such deviations should be reported transparently — they add scientific value by revealing the limits of the assumed model.
Sources
- Morey, R. D., Chambers, C. D., Aitken, M. R. F., Harris, C. R., Hoekstra, R., Lakens, D., Lewandowsky, S., Morey, C. C., Newman, D. P., Schonbrodt, F. D., Vanpaemel, W., Wagenmakers, E. J., & Zwaan, R. A. (2022). The Peer Reviewers' Openness Initiative: Incentivising open research practices through peer review. Royal Society Open Science, 3(1), 150547. link ↗
- Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102. DOI: 10.1002/sim.8086 ↗
How to cite this page
ScholarGate. (2026, June 3). Simulation-Assisted Confirmatory Research. ScholarGate. https://scholargate.app/en/research-design/simulation-assisted-confirmatory-research
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Confirmatory factor analysisPsychometrics↔ compare
- MONTE-CARLO-SIMULATIONDecision-making↔ compare
- Power analysisStatistics↔ compare
- Randomized Controlled TrialExperimental design↔ compare
- Structural Equation ModelingResearch Statistics↔ compare