Философия науки

Science is a systematic, evidence-based, and self-correcting enterprise for producing reliable knowledge about the world. It is distinguished from everyday belief, dogma, and pseudoscience by empirical testing, theoretical framing, objectivity-as-intersubjectivity, and the social and institutional character of knowledge production. Defining science by a single method proves insufficient; science must be understood as both a practical and a communal activity.

The scientific method is an epistemic framework that enables questions about nature to be answered systematically. It is typically described as a cyclical process comprising question formulation, hypothesis generation, prediction, controlled observation or experimentation, and the analysis and revision of findings. There is, however, no single fixed algorithm; the method varies considerably by discipline, research context, and epistemic goal. Testability, reproducibility, and inferential validity are widely regarded as the core criteria by which scientific practice is evaluated.

The problem of induction, posed by David Hume, challenges the logical foundation of reasoning from past observations to future events. Hume argued that inductive inference can be justified neither by pure logical necessity nor by experience without circular reasoning. This problem fundamentally unsettles the certainty claims of scientific laws and has directly motivated Karl Popper's falsificationist response and Bayesian approaches to confirmation and rational belief.

The hypothetico-deductive method is a central procedure in scientific inquiry. A hypothesis is proposed, observable consequences are logically deduced from it, and these predictions are then subjected to empirical test. Confirming instances lend support to the hypothesis without proving it conclusively, while a failed prediction counts as evidence against it. The method is explicitly contrasted with naive inductivism and forms the backbone of much contemporary philosophy of science and experimental practice.

The demarcation problem asks what criterion distinguishes genuine science from pseudoscience and metaphysics. Karl Popper proposed falsifiability as the key criterion: scientific claims must be testable and refutable, whereas unfalsifiable claims fall outside science. He drew on astrology and certain interpretations of psychoanalysis as illustrative examples. Later philosophers argued that demarcation is better understood as a multi-criterial and graded problem rather than a single sharp boundary.

Positivism is a philosophical position holding that genuine knowledge can only be derived from observable facts and the lawlike regularities connecting them. Systematized by Auguste Comte, the view excludes metaphysical and theological explanations from the domain of science. Comte's 'law of three stages' argues that human thought progresses through theological, metaphysical, and positive (scientific) stages. Positivism profoundly influenced the social sciences in their pursuit of general laws and laid conceptual groundwork for later logical positivism, from which it should nonetheless be carefully distinguished.

Logical positivism is an influential movement in the philosophy of science developed in Vienna during the 1920s by a group of philosophers known as the Vienna Circle, led by Moritz Schlick. Its central claim is that a statement is cognitively meaningful only if it is empirically verifiable or analytically true. Metaphysical propositions, failing this criterion, are dismissed as meaningless. By combining formal logic with empiricism and championing the unity of science under a common language, logical positivism profoundly shaped subsequent debates in philosophy of science.

Falsificationism is a philosophy of science principle developed by Karl Popper. According to Popper, scientific theories can never be conclusively verified; they can only be falsified. Science advances through bold conjectures subjected to severe tests, and a theory that cannot in principle be refuted is not scientific. This principle offers a deductive solution to Hume's problem of induction and serves as the central criterion for demarcation — distinguishing genuinely scientific claims from non-scientific ones.

Thomas S. Kuhn argued that science does not progress through simple linear accumulation of knowledge. Instead, it alternates between periods of 'normal science' — routine puzzle-solving within an accepted paradigm — and revolutionary episodes in which an anomaly-laden paradigm is overthrown and replaced. Through the concepts of incommensurability, theory-ladenness of observation, and paradigm shift, Kuhn transformed how historians and philosophers understand scientific change, challenging the prevailing Popperian image of science as continuous falsification.

Imre Lakatos developed the methodology of scientific research programmes to reconcile Karl Popper's falsificationism with Thomas Kuhn's historical account of science. In this framework, every programme consists of an unfalsifiable 'hard core' and a surrounding 'protective belt' of auxiliary hypotheses. Programmes that predict novel facts are called 'progressive', while those that only accommodate anomalies after the fact are 'degenerating'. Theory appraisal depends on cumulative development over time rather than on any single experimental test.

In Against Method (1975), Paul Feyerabend argues that there is no universal, fixed scientific method. His slogan "anything goes" is not a rejection of science but a provocation against methodological monism. Feyerabend shows that major scientific advances — most notably Galileo's — were achieved by violating the methodological canons of the time. Science thrives on theoretical pluralism and the proliferation of competing ideas; adherence to a single methodological dogma would stifle rather than promote scientific progress.

Scientific realism holds that successful theories are approximately true and that unobservable entities such as electrons and genes genuinely exist. Anti-realism challenges this: Bas van Fraassen's constructive empiricism accepts theories as empirically adequate without committing to the existence of unobservables. Instrumentalism treats theories as practical tools rather than truth-bearing descriptions. The debate reflects a foundational philosophical question about the ultimate aim of science and the limits of what scientific inquiry can rightfully claim to know about reality.

The deductive-nomological (DN) model, developed by Carl Hempel and Paul Oppenheim in 1948, holds that to explain an event is to deduce it logically from general laws and initial conditions. Also known as the covering-law model, it treats explanation and prediction as structurally symmetric operations. The model was supplemented by an inductive-statistical (IS) variant for probabilistic laws, yet it faced serious challenges from objections concerning asymmetry and relevance, giving rise to causal and unificationist alternatives.

Bayesian philosophy of science treats scientific inference as probabilistic belief updating. A prior degree of belief in a hypothesis is revised in light of new evidence via Bayes' theorem, yielding a posterior probability. On this view, evidence confirms a hypothesis precisely when it raises the hypothesis's probability. The framework offers a graded, quantitative response to the problem of induction and the logic of confirmation, though the choice of subjective prior probabilities remains its central point of contention.

A research paradigm is the set of foundational assumptions a researcher holds about the nature of reality, the relationship between the knower and the known, and how knowledge should be pursued. Guba and Lincoln's influential framework classifies paradigms along ontological, epistemological, and methodological axes. Positivism assumes an objective reality and favours quantitative methods; interpretivism and constructivism foreground socially constructed meanings and qualitative inquiry; pragmatism prioritises what works, supporting mixed methods; critical realism investigates unobservable generative mechanisms. Together these paradigms form the bridge between philosophy of science and concrete methodological choices.

The reproducibility crisis has profoundly shaken the scientific community, particularly in psychology and biomedicine. A substantial proportion of published findings have failed to replicate in independent attempts, drawing attention to p-hacking, publication bias, low statistical power, and questionable research practices. The open-science movement responds with remedies including preregistration, registered reports, open data and code sharing, and emphasis on larger sample sizes. From a philosophy of science perspective, the crisis has foregrounded the limits of scientific validation and the need to restructure research infrastructure.

Statistical hypothesis testing is an inferential framework for evaluating testable claims about a population using data. The null hypothesis (H₀) asserts "no effect"; the alternative hypothesis (H₁) asserts there is one. The researcher assumes H₀, measures how surprising the observed data are under that assumption, and either rejects H₀ or fails to reject it. "Failing to reject" is never equivalent to "accepting" H₀, and H₁ is never proven. Fisher's significance testing and the Neyman-Pearson decision framework rest on distinct philosophical foundations.

The Central Limit Theorem states that the distribution of the sum (or mean) of independent, identically distributed random variables with finite variance converges to a normal distribution as sample size grows, regardless of the shape of the underlying population. This makes it the theoretical cornerstone of normal-based statistical inference, including confidence intervals and hypothesis tests.

Sampling is the process of selecting a subset of a population to collect data and draw conclusions about the whole. Probability sampling methods — simple random, systematic, stratified, cluster, and multistage — assign every unit a known, nonzero selection probability, enabling unbiased estimation and statistical generalization. Non-probability methods — convenience, purposive, quota, and snowball sampling — are cheaper and sometimes necessary, but are prone to selection bias and limit the validity of statistical inference.

The p-value expresses the probability of obtaining data at least as extreme as observed, given that the null hypothesis is true. Researchers compare it to a pre-chosen significance level α (commonly 0.05) to decide whether to reject H₀. However, statistical significance does not guarantee practical importance; a p-value can be small even for a trivially small effect. The American Statistical Association's 2016 statement emphasizes that p-values should not be used as the sole basis for scientific or policy decisions.

In statistical hypothesis testing, two fundamental error types exist. A Type I error (α) occurs when a true null hypothesis is rejected — also called a false positive. A Type II error (β) occurs when a false null hypothesis is not rejected — a false negative. Statistical power (1 − β) is the probability of detecting a true effect. A trade-off governs these errors: reducing α increases β and lowers power, while larger samples and larger effect sizes raise power.

Errors threatening measurement accuracy in research fall into two main categories: sampling error and non-sampling error. Sampling error is random variation arising from observing only a subset of a population; it shrinks as sample size increases. Non-sampling errors encompass coverage, nonresponse, measurement, and processing problems that larger samples do not reduce. Bias is systematic deviation from the true value, while variance is random spread around an estimate. The total survey error framework organizes all these error sources into a coherent structure for evaluation and reduction.

A confidence interval expresses a parameter estimate as a range of plausible values rather than a single point. A 95% confidence interval means that, over many repeated samples, 95% of the intervals constructed by the same procedure would contain the true parameter. This confidence is a property of the procedure, not of any single computed interval. The width reflects estimation precision, while the location reflects the magnitude of the effect.

Effect size is a statistical measure that quantifies the magnitude of a phenomenon independently of sample size. Because p-values conflate effect magnitude with sample size, they are insufficient for judging practical significance. Indices such as Cohen's d, Pearson r, odds ratio, and eta-squared provide researchers with a standardized metric for assessing both practical importance and the essential input for meta-analysis. Conventional small/medium/large benchmarks are rough heuristics, not universal thresholds.

Descriptive statistics summarize and display the data at hand through measures of central tendency, dispersion, tables, and graphs, making no reference to any population beyond the observed cases. Inferential statistics use a sample together with probability models to draw conclusions about a wider population — the core tools being estimation, hypothesis testing, and confidence intervals. The leap from description to inference hinges critically on how the sample was obtained.

The level of measurement of a variable determines which mathematical operations and statistical techniques are meaningful for that variable. Stevens (1946) classified measurement scales as nominal, ordinal, interval, and ratio. This hierarchy serves as a fundamental guide for choosing analyses, though it is not an absolute rule; researchers must always question which statistics are genuinely interpretable.

Reliability refers to the consistency or repeatability of a measurement across time points, raters, or items. Validity concerns whether an instrument actually measures what it claims to measure. A measure can be reliable yet invalid — consistently off-target — while validity presupposes at least some degree of reliability. Together these two properties form the foundational quality criteria of scientific measurement.

A correlation between two variables does not establish that one causes the other. Confounding variables, reverse causation, selection effects, and chance can all produce statistical association without any genuine causal link. Establishing causation requires randomized controlled experiments or causal-inference designs such as instrumental variables, difference-in-differences, or regression discontinuity, along with explicit and testable assumptions about the underlying data-generating process.

Measures of central tendency summarise the centre of a distribution with a single value. The arithmetic mean uses all observations; the median is the middle value of an ordered dataset; the mode is the most frequently occurring value. Which measure to choose depends on the level of measurement and the shape of the distribution. When outliers are present, the median provides a more robust estimate of the centre than the mean.

While measures of central tendency summarize the typical value in a dataset, measures of dispersion quantify how much the values spread around that centre. Range, interquartile range (IQR), variance, standard deviation, and coefficient of variation are the principal statistics serving this purpose. Standard deviation is the square root of variance and returns to the original unit of measurement. IQR is resistant to outliers; the coefficient of variation enables comparison of spread across variables measured on different scales. Dispersion is as informative for research as the centre.

Skewness measures the asymmetry of a distribution: positive skewness indicates a long right tail, while negative skewness indicates a long left tail. Kurtosis describes the degree to which a distribution is more peaked or flatter than the normal, capturing tailedness. Together, these two statistics reveal how far data depart from normality, which directly affects the validity of parametric tests and the appropriate choice of central-tendency measure.

Probability is a mathematical framework that quantifies uncertainty with values between 0 and 1. Its building blocks are the sample space, events, the addition rule, the multiplication rule, conditional probability, and Bayes' theorem. Random variables map outcomes to numbers and underpin every statistical model. A firm grasp of probability is a prerequisite for using statistical tools such as hypothesis testing, confidence intervals, and Bayesian updating in a principled and meaningful way.

The normal distribution is a symmetric bell curve defined entirely by its mean and standard deviation. Approximately 68%, 95%, and 99.7% of values fall within 1, 2, and 3 standard deviations of the mean — the empirical rule. Standardizing an observation via z = (x − μ)/σ places it on the standard normal scale, enabling comparisons across different measurement scales. Thanks to the central limit theorem, sample means converge to normality in large samples, making the normal distribution the backbone of classical statistical inference.

Beyond the normal distribution, researchers rely on several fundamental probability distributions to model distinct data-generating processes. The t-distribution handles mean comparisons in small samples, the chi-square addresses variance tests and categorical data, the F-distribution covers variance ratios and ANOVA, the binomial counts successes in binary trials, and the Poisson models rare-event frequencies. Each distribution provides the mathematical foundation for a family of inferential tests.

A sampling distribution is the distribution of a statistic — such as the sample mean — computed across all possible samples of a given size drawn from a population. Its spread is measured by the standard error. Hypothesis tests and confidence intervals are statements about where an observed statistic falls within its sampling distribution; the central limit theorem describes the shape of that distribution. The sampling distribution is therefore the conceptual bridge that makes statistical inference possible.

Standard deviation (SD) measures the spread of individual data points; standard error (SE) describes how precisely a statistic, such as the sample mean, estimates the population parameter. SE = SD / sqrt(n), so SE decreases as the sample grows while SD remains stable. Confusing the two is a common and consequential error in research reporting, leading to misleading representations of variability and precision.

Degrees of freedom (df) count the number of values in a dataset that are free to vary once constraints have been imposed by estimated parameters. Estimating the mean from a sample of n observations leaves only n−1 values free to vary independently, which is why sample variance is divided by n−1. The df value determines the precise shape of the t, chi-square, and F distributions, and therefore controls the critical values that decide the outcomes of statistical tests.

Parametric tests (t-test, ANOVA, Pearson r) assume that the data follow a specific distributional form — typically normality — and operate on raw numerical values. Non-parametric tests (Mann–Whitney, Wilcoxon, Kruskal–Wallis, Spearman) are rank-based and impose far fewer distributional assumptions, making them suitable for small samples, ordinal data, or violated assumptions — at some cost in statistical power when parametric assumptions genuinely hold.

Most classical statistical tests rest on specific conditions: normality of residuals, equality of group variances (homoscedasticity), independence of observations, and linearity (in regression). When these assumptions are violated, estimates may be biased or p-values may become invalid. Researchers inspect assumptions through diagnostic plots and formal tests, and respond with data transformations, robust methods, or non-parametric alternatives when violations are detected.

A two-tailed test asks whether a parameter differs in either direction and splits the significance level α equally across both tails of the distribution. A one-tailed test concentrates all of α in one pre-specified direction, gaining statistical power there but remaining blind to effects in the opposite direction. One-tailed tests are justified only when a directional hypothesis is explicitly stated before examining the data; choosing the tail post hoc inflates the Type I error rate.

When many hypothesis tests are conducted in a single study, the probability of obtaining at least one false positive by chance rises far above the nominal α level. This phenomenon is captured by the concept of the family-wise error rate (FWER). The Bonferroni correction controls FWER by dividing α by the number of tests; the Benjamini–Hochberg procedure instead controls the false discovery rate (FDR), offering greater statistical power in large-scale testing scenarios.

Resampling methods estimate sampling variability empirically from the data itself rather than from closed-form formulas. The bootstrap repeatedly samples observations with replacement to build a distribution for a statistic; permutation tests reshuffle group labels to construct a null distribution; the jackknife removes one observation at a time. These techniques are invaluable when analytic formulas are intractable or parametric distributional assumptions are in doubt, making them broadly applicable across diverse research contexts.

Frequentist and Bayesian inference are the two dominant philosophies of statistics, differing in how they define probability and treat parameters. The frequentist approach treats parameters as fixed and probability as long-run frequency, using p-values and confidence intervals as its main tools. The Bayesian approach treats parameters as uncertain quantities, combining prior beliefs with observed data via likelihood to produce a posterior distribution. The two frameworks answer subtly different questions and are increasingly used together in modern research practice.

Statistical power (1 − β) is the probability of detecting an effect that truly exists. Sample size, effect size, significance level, and variance are the four key determinants of power. Underpowered studies miss real effects and produce unreliable, exaggerated estimates. An a priori power analysis calculates the sample size needed for adequate power before data collection begins, with 0.80 commonly adopted as the target threshold.

Choosing the right statistical test depends on the structure of the research question and the characteristics of the data. Key factors include the level of measurement of the outcome variable, the number of groups, whether samples are independent or paired, and whether distributional assumptions are met. Two independent means call for an independent t-test; three or more groups for ANOVA; the association between two continuous variables for correlation or regression; and categorical data for a chi-square test. Selecting the wrong test leads to misleading conclusions.

Missing values are inevitable in research data; what matters is why the data are missing. Statistical literature classifies missingness into three mechanisms: Missing Completely At Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR). The mechanism in play directly determines which analytic strategy will yield valid results. Simple approaches such as listwise deletion appear convenient but often produce biased estimates; multiple imputation and maximum-likelihood methods are the modern standard and are statistically valid under MAR.

Outliers are observations that lie far from the bulk of a distribution. Influential observations are points that disproportionately alter a model's parameter estimates; not every outlier is influential, and not every influential point appears extreme. Detection relies on z-scores, the IQR rule, and — in regression — leverage, standardized residuals, and Cook's distance. Outliers should never be deleted reflexively; each case must be investigated to determine whether it reflects a data error or a genuinely important signal.

Data transformation reshapes raw measurements so that statistical assumptions are met or scales become comparable. Log and square-root transforms reduce right skew and stabilize variance; the Box–Cox family generalizes this approach. Z-standardization shifts data to a distribution with zero mean and unit variance, while min-max scaling constrains values to a fixed range. Because transformations alter the interpretation of results, they must always be reported and, where necessary, reversed before drawing substantive conclusions.

Sensitivity is the proportion of true positives a test correctly identifies — how many people with the condition receive a positive result. Specificity is the proportion of true negatives correctly identified — how many people without the condition receive a negative result. A highly sensitive test rarely misses cases; a highly specific test rarely raises false alarms. There is an inherent trade-off between these two measures, controlled by the decision threshold, and researchers must consider which type of error carries the greater practical cost.

Positive predictive value (PPV) is the probability that someone with a positive test result truly has the condition; negative predictive value (NPV) is the probability that a negative result is truly negative. Unlike sensitivity and specificity, predictive values depend heavily on prevalence — the background rate of the condition in the population. The same test yields a much lower PPV in a low-prevalence population, a fact frequently overlooked by researchers interpreting diagnostic or screening results.

A Receiver Operating Characteristic (ROC) curve plots the true-positive rate against the false-positive rate across all possible thresholds, revealing the trade-off between correctly catching positives and generating false alarms. The Area Under the Curve (AUC) summarizes this discrimination ability in a single number ranging from 0.5 (chance) to 1.0 (perfect). Formally, AUC equals the probability that the model assigns a higher score to a randomly chosen positive instance than to a randomly chosen negative instance.

The confusion matrix cross-tabulates a classification model's predictions against the true class labels in a compact four-cell table: true positives, false positives, false negatives, and true negatives. Nearly every standard classification metric — accuracy, precision, recall, specificity, and F1 score — is derived from these four values. Reading the matrix directly reveals which kinds of errors a model makes, exposing information that a single accuracy figure often conceals, especially when class distributions are imbalanced.

Summarising a classification model with a single number is often misleading. Accuracy hides true performance when classes are imbalanced. Precision measures how many predicted positives are truly positive; recall measures how many real positives were actually caught. The F1 score is their harmonic mean, balancing the two. Which metric to use depends on whether false positives or false negatives carry the higher cost in a given problem.

A likelihood ratio (LR) is a single number that summarizes how much a test result changes the pre-test probability of a condition. The positive LR quantifies how much the odds increase when the test is positive; the negative LR shows how much the odds decrease when the test is negative. By combining sensitivity and specificity into one prevalence-independent measure, LRs allow clinicians and researchers to move from pre-test to post-test probability using Bayes' theorem in a straightforward and transparent way.

Cohen's d expresses the difference between two group means in standard-deviation units, making effect magnitude comparable across studies that use different scales or measures. While statistical significance tells us whether a difference exists, Cohen's d tells us how large that difference is. Cohen's rough benchmarks of 0.2 small, 0.5 medium, and 0.8 large are conventions and starting points, not universal laws, and must always be interpreted in the context of the research domain.

Variance-explained effect sizes express, as a proportion between 0 and 1, how much of the total variability is accounted for by the model. In ANOVA designs, eta-squared and partial eta-squared serve this purpose; omega-squared is a less biased alternative. In regression, R-squared and adjusted R-squared play the same role. A significant p-value only confirms that an effect exists; these measures reveal how large that effect is in practical terms.

The odds ratio (OR) compares the odds of an outcome between two groups. The odds in the exposed group are divided by the odds in the unexposed group. OR = 1 indicates no association; values above 1 indicate higher odds with exposure. It is the natural output of logistic regression and case-control studies. A frequent misconception treats it as equivalent to relative risk, which holds only when the outcome is rare.

Relative risk (risk ratio) divides the probability of an outcome in an exposed group by that in an unexposed group, expressing how many times more likely the outcome is. The risk difference (absolute risk reduction) subtracts the two probabilities, showing the absolute size of the change. Relative measures can look dramatic while the absolute change is tiny, so both should be reported together for a complete and honest interpretation.

The number needed to treat (NNT) expresses how many patients must receive a treatment for one additional beneficial outcome to occur. It is calculated as the reciprocal of the absolute risk reduction (ARR): NNT = 1/ARR. Because it converts relative effects into a tangible clinical quantity, it is widely adopted by clinicians and policy makers. A small NNT signals an efficient treatment, while a large NNT indicates marginal benefit.

The hazard ratio (HR) compares the instantaneous rate of an event between two groups across follow-up time. It is the central output of Cox proportional-hazards survival models. HR = 1 means no difference; values above 1 indicate a higher event rate in the exposed group. Valid interpretation requires verifying the proportional hazards assumption. The HR is not an absolute risk measure; it is the quotient of two hazard functions, not a proportion.

A significant chi-square test establishes that an association exists, but it says nothing about how strong the association is. Effect-size measures fill this gap. The phi coefficient is used for 2x2 tables, Cramér V for larger tables, the contingency coefficient as a compatible alternative for either table type, and the point-biserial correlation when one variable is binary and the other is continuous. These measures rescale the association onto a standard metric so that practical importance, not just statistical significance, can be assessed and communicated.

Overfitting occurs when a model memorizes noise and incidental patterns in the training data, losing the ability to generalize to new observations. Underfitting is the opposite extreme: the model is too simple to capture the true underlying relationship and performs poorly everywhere. Both conditions undermine predictive validity. Researchers detect these imbalances by comparing training and validation or test performance, then adjust model complexity accordingly.

Prediction error decomposes into three components: bias, variance, and irreducible noise. Bias arises from overly simplistic assumptions a model makes about the true relationship; models that are too simple underfit and produce systematic error. Variance reflects sensitivity to the particular training sample; overly complex models overfit and behave inconsistently across datasets. Because bias and variance move in opposite directions as model complexity changes, the best-performing model sits at an intermediate level of complexity that minimises total expected error.

To estimate how a model performs on unseen data, the dataset is divided into three parts: a training set to fit the model, a validation set to tune hyperparameters and select among competing models, and a held-out test set used only once for final evaluation. Cross-validation rotates these splits to use data more efficiently. Data leakage — allowing test information to influence training — silently inflates performance estimates and is a critical error that must be avoided.

Regularization adds a penalty on model complexity to the fitting objective, shrinking coefficients toward zero to reduce variance and overfitting. L2 (ridge) shrinks all coefficients smoothly; L1 (lasso) can set some exactly to zero, performing variable selection; elastic net blends both. A tuning parameter controls the strength of the penalty, trading a little bias for a large reduction in variance.

When comparing statistical models, relying on goodness-of-fit alone is misleading because more complex models always fit better. Information criteria balance how well a model fits the data against the number of parameters it uses, thereby guarding against overfitting. AIC and BIC operationalize this trade-off in different ways; lower values indicate a better model. Together they formalize the principle of parsimony, rewarding simpler explanations that still capture the signal in the data.

Goodness-of-fit measures quantify how closely a model reproduces observed data. For continuous outcomes, R-squared reports variance explained, while RMSE, MAE, and MAPE summarize prediction error in interpretable units. For models fit by likelihood, deviance and the chi-square goodness-of-fit test apply. Good fit on training data does not guarantee good prediction on new data — error must always be judged out of sample.

Regression diagnostics systematically examine whether the assumptions of a fitted model actually hold. Residual plots expose nonlinearity and heteroscedasticity; Q-Q plots assess whether residuals follow a normal distribution; the Durbin–Watson statistic detects autocorrelation; the variance inflation factor (VIF) measures multicollinearity; and leverage together with Cook's distance identify influential observations. Violations can bias coefficient estimates and distort standard errors.

An interaction effect occurs when the effect of one predictor on an outcome depends on the level of another predictor — the relationship is not simply additive. Modeled by including a product term in a regression equation, interactions are the statistical foundation of moderation analysis. Correct interpretation requires examining simple slopes at meaningful values of the moderator, and centering continuous predictors before forming the product term makes the resulting main effects and interaction coefficient far more interpretable.

A random variable is a mathematical tool that assigns a numerical value to every possible outcome of a probabilistic process. Discrete random variables take countable values described by a probability mass function; continuous ones take values over a range described by a probability density function. In both cases, the cumulative distribution function gives the probability that the variable falls at or below a given threshold. The concept transforms uncertainty into computable mathematics.

The expected value E[X] is the long-run average of a random variable, computed as a probability-weighted sum of its possible values. The variance Var[X] measures spread around that mean as the expected squared deviation. Its square root, the standard deviation, shares the same units as the original variable. Together, these two statistics form the first two moments that summarise a distribution's location and dispersion. Correctly computing and reporting them strengthens the credibility and interpretability of research findings.

The law of large numbers states that as sample size grows, the sample mean converges to the true expected value of the population. It explains why larger samples yield more reliable estimates and why casinos and insurers profit predictably over many trials. The law concerns where the average settles; the central limit theorem, by contrast, describes the shape of its fluctuations around that settled value.

Bayes' theorem describes how to update the probability of a hypothesis when new evidence arrives. The core logic is that the posterior probability is proportional to the likelihood multiplied by the prior probability. The theorem explains why a positive result on a test for a rare disease can still leave the probability of actually having that disease surprisingly low, a phenomenon known as the base-rate effect. It is the mathematical foundation of Bayesian inference, diagnostic reasoning, and likelihood ratios.

A joint distribution describes the probabilities of combinations of two or more variables simultaneously. Summing or integrating out the remaining variables yields the marginal distribution of a single variable. Fixing one variable at a specific value produces the conditional distribution of the other. When the conditional equals the marginal, the variables are independent. Covariance and correlation summarize how variables vary together, quantifying the direction and strength of their linear relationship.

Hypothesis testing is a six-step decision process by which a researcher systematically evaluates a claim against data. The steps are: state the hypotheses, choose a significance level and appropriate test, verify the test assumptions, compute the test statistic and p-value, decide whether to reject the null hypothesis, and interpret the result in context together with an effect size and confidence interval. Reporting the p-value alone is never sufficient.

Every statistical test assumes a set of conditions holds in the data. These assumptions — normality, homogeneity of variance, independence, and linearity — when unmet can lead to invalid conclusions. Researchers must check these conditions systematically before analysis using both visual diagnostics and formal tests, and must respond to violations with data transformation, robust methods, or non-parametric alternatives rather than ignoring the problem.

Reporting statistical results fully means providing everything a reader needs to independently evaluate the evidence: the name of the test, the test statistic, degrees of freedom, and the exact p-value, together with — not instead of — an effect size and its confidence interval. Means and standard deviations accompany group comparisons; tables and figures avoid duplication. Style guides such as the APA Publication Manual standardize this format across disciplines.

A p-value alone is a weak basis for conclusions. Read together, the three indicators tell a fuller story: the p-value indicates whether an observed effect is distinguishable from chance; the effect size states how large or meaningful that effect is; and the confidence interval reveals the precision of the estimate and the range of plausible values. In large samples, a tiny and unimportant effect can appear statistically significant; in small samples, a genuine and meaningful effect may fail to reach the significance threshold.