Common Probability Distributions

Core Idea: Each Distribution Has a Data-Generating Story

A probability distribution defines the possible values a random variable can take and assigns probabilities to those values. Researchers move beyond the normal distribution because each data-generating process has its own mathematical structure. Means in small samples follow the t-distribution, whereas event counts follow the Poisson. Choosing the correct distribution is the first step in avoiding an incorrect test statistic, a misinterpreted p-value, or misleading conclusions. Statistical inference therefore begins with the model assumptions implied by the process that produced the data.

Continuous Distributions: t, Chi-Square, and F

The t-distribution is used for mean comparisons when the population variance is unknown and the sample is small. The standard error is computed as SE = SD / sqrt(n); as n grows the t-distribution approaches the normal. The chi-square distribution is derived from the sum of squared standard normal variables and underlies tests of independence and goodness of fit. The F-distribution is the ratio of two independent chi-square variables and forms the basis for comparing group variances in ANOVA. All three distributions are shaped by degrees of freedom (df); smaller df produce heavier tails.

Discrete Distributions: Binomial and Poisson

The binomial distribution models the number of successes in a fixed number n of trials, each with success probability p. Trials must be independent — an assumption frequently violated in real research. The Poisson distribution models the count of rare events in a fixed unit of time or space; its single parameter lambda (λ) is the average event rate per unit. The key assumptions are that events occur independently and that the mean rate remains constant. Both distributions form the foundation for advanced methods such as logistic regression and count-data models.

Common Misuses and Importance in Research Practice

A common error is automatically replacing the t-test with a z-test as the sample grows, or applying chi-square to cells with very low expected frequencies — Fisher's exact test is preferred in that case. In Poisson models, ignoring overdispersion can lead to underestimated standard errors. Failing to verify the independence assumption in a binomial test produces misleading p-values. Selecting the correct distribution is not merely a mathematical preference but a critical step for the validity of research findings. Reporting model assumptions is an integral part of transparent scientific practice.

Sources