Joint, Marginal, and Conditional Distributions
How several variables behave together
A joint distribution describes the probabilities of combinations of two or more variables simultaneously. Summing or integrating out the remaining variables yields the marginal distribution of a single variable. Fixing one variable at a specific value produces the conditional distribution of the other. When the conditional equals the marginal, the variables are independent. Covariance and correlation summarize how variables vary together, quantifying the direction and strength of their linear relationship.
Core Concepts and Formulas
For two discrete variables X and Y, the joint probability P(X=x, Y=y) gives the probability of each value combination occurring. The marginal distribution is obtained by summing over the other variable: P(X=x) = sum_y P(X=x, Y=y). The conditional distribution describes the distribution of one variable given knowledge of the other: P(Y=y | X=x) = P(X=x, Y=y) / P(X=x). For continuous variables, sums become integrals but the logic is identical. Independence holds when P(X,Y) = P(X) x P(Y) for all values.
How to Compute and Read These Distributions
For discrete variables, a cross-tabulation (contingency table) displays the joint distribution directly; row and column totals give the marginal distributions. The conditional distribution is obtained by dividing each row or column by its marginal total. For continuous variables, density plots such as 2D histograms or contour plots visualize the joint distribution; the marginal densities appear as projections onto each axis. Covariance cov(X,Y) = E[(X-mu_X)(Y-mu_Y)] and correlation r = cov(X,Y)/(sigma_X sigma_Y) summarize the direction and strength of linear co-variation.
Common Misuses and Misconceptions
The most common error is confusing marginal and conditional distributions; this confusion underlies Simpson paradox, where a trend observed in every subgroup can reverse when groups are combined. A second frequent mistake is equating zero correlation with independence; two variables can show zero correlation while having a strong nonlinear relationship. Third, researchers sometimes treat P(A|B) and P(B|A) as interchangeable, a fallacy that drives the prosecutor fallacy in forensic statistics. Writing the joint distribution as a product of marginals is valid only under the independence assumption and should not be applied carelessly.
Why It Matters and How to Report It
Understanding joint distributions is foundational for multivariate analyses including regression, structural equation modeling, and tests of multivariate normality. Ignoring the dependence structure among variables leads to incorrect standard errors and biased coefficient estimates. When reporting results, present a correlation matrix, scatterplot matrix, and where relevant a table of conditional means so readers can assess the joint structure. In contingency tables, reporting row percentages and column percentages separately clarifies which conditioning is being applied and improves the transparency and replicability of interpretations.