Null Hypothesis Testing
Null Hypothesis Significance Testing (NHST) and Hypothesis Formulation · Also known as: NHST, hypothesis formulation, null hypothesis, alternative hypothesis, two-tailed test
Null Hypothesis Significance Testing (NHST) is the dominant statistical framework in empirical research. The null hypothesis (H₀) represents the default assumption—typically 'no effect' or 'no difference'—while the alternative hypothesis (H₁) represents the claim being tested. The test calculates the probability of observing the data given H₀ is true (p-value); if p is very small, H₀ is rejected in favor of H₁. Formulated by Ronald Fisher and extended by Neyman and Pearson in the early 20th century, NHST is foundational to confirmatory research but has been widely critiqued for misuse and misinterpretation.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use NHST when: (1) You have a clear, pre-specified hypothesis before collecting data (confirmatory research). (2) You want to control the false positive rate (Type I error) at a specified level (α). (3) Your research question naturally frames a null hypothesis: 'Does this treatment differ from placebo?' rather than 'How much does this treatment help?'. Do NOT use NHST for exploratory research without pre-registration, as multiplicity and selection bias inflate false positives. Combine NHST with effect sizes, confidence intervals, and consideration of practical significance for comprehensive inference.
Strengths & limitations
- Provides a principled, standardized framework for decision-making in research, enabling reproducibility and comparison across studies.
- Controls Type I error rate (false positive probability) through the pre-specified significance level, protecting scientific integrity.
- Well-integrated with software, statistical textbooks, and discipline-specific standards, making it widely accessible.
- Enables pre-specification of hypotheses a priori, reducing p-hacking and selective reporting when done properly.
- The null hypothesis (usually 'no effect') is often implausible; in large samples, any trivial effect becomes statistically significant.
- Binary decision framework (reject/fail to reject at α threshold) obscures the continuous nature of evidence and promotes dichotomous thinking.
- Does not directly answer practical questions: 'Does an effect exist?' is different from 'Is the effect large enough to matter?'
- Assumes a fixed sample size; optional stopping or peeking at data invalidates p-values (though pre-registration mitigates this).
- Prone to misuse (p-hacking, HARKing, undisclosed flexibility) in the absence of strong research norms and enforcement.
Frequently asked
Why do we test the null hypothesis instead of testing whether the alternative hypothesis is true?
The null hypothesis (e.g., 'no effect') is specific and easy to test mathematically: it specifies an exact parameter value (e.g., μ = 0). The alternative hypothesis is often vague about magnitude (e.g., 'an effect exists'). By testing H₀ and calculating the probability of the data under H₀, we can make a principled decision. This is a matter of mathematical convenience and philosophical tradition, not absolute logical necessity.
Should I use a one-tailed or two-tailed test?
Use two-tailed tests by default unless you have a strong theoretical or prior reason to expect an effect in one direction only. Two-tailed tests are more conservative and protect against unexpected findings in the opposite direction. If you use one-tailed, pre-specify it before data analysis. Post-hoc switching to one-tailed tests is p-hacking.
What does 'fail to reject the null hypothesis' mean?
It means the data do not provide sufficient evidence to reject H₀ at your chosen significance level. It does NOT mean H₀ is true, that the effect does not exist, or that you have accepted H₀. Low power or small sample size can result in failure to reject a false H₀. Always report effect sizes and confidence intervals to clarify the magnitude of observed effects, even if they are not statistically significant.
Is p = 0.049 statistically significant at α = 0.05?
Yes, by the strict α = 0.05 threshold. However, p = 0.049 and p = 0.051 carry nearly identical evidence; the binary threshold creates a false cliff. This is why many researchers now report actual p-values and effect sizes rather than binary 'significant/non-significant' classifications. Consider the practical significance and replicate findings before drawing strong conclusions near the α boundary.
Can I use hypothesis testing for exploratory research?
Not without adjusting for multiple comparisons or pre-registration. In exploratory research, you test many hypotheses, inflating the false positive rate (family-wise error). To use NHST exploratorily: (1) Pre-register your hypotheses before seeing data. (2) Conduct confirmatory tests on a separate dataset. (3) Report exploratory analyses as exploratory and adjust p-values (Bonferroni, FDR correction). The gold standard is two-stage research: exploratory phase generates hypotheses, confirmatory phase tests them on new data.
Sources
- Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver and Boyd. link ↗
- Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society, 231, 289–337. DOI: 10.1098/rsta.1933.0009 ↗
- Gigerenzer, G., & Marewski, J. N. (2015). Surrogate Science: The Idol of a Universal Method for Scientific Inference. Journal of Management, 41(2), 421–440. DOI: 10.1177/0149206314547522 ↗
How to cite this page
ScholarGate. (2026, June 3). Null Hypothesis Significance Testing (NHST) and Hypothesis Formulation. ScholarGate. https://scholargate.app/en/research-statistics/null-hypothesis
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Confidence IntervalResearch Statistics↔ compare
- P-Value and Statistical SignificanceResearch Statistics↔ compare
- Statistical Power and Sample SizeResearch Statistics↔ compare
- Type I and Type II ErrorsResearch Statistics↔ compare