Ordinary Least Squares (OLS)

Under five classical assumptions — linearity, full rank X, zero conditional mean errors, homoskedasticity, and uncorrelated errors — the Gauss-Markov theorem proves that the OLS estimator has the smallest variance among all estimators that are both linear in y and unbiased. This is the 'BLUE' property. It does not require the errors to be normally distributed; that additional assumption is needed only for exact finite-sample t- and F-inference.

When the homoskedasticity assumption is in doubt — that is, when the variance of the errors may differ across observations — conventional OLS standard errors are incorrect, even though the coefficient estimates β̂ remain consistent. Heteroskedasticity-robust (White) standard errors correct the estimated covariance matrix without requiring a specific model for the variance. They are now routinely used in applied econometrics as a precaution even when heteroskedasticity is not strongly suspected.

When the errors are assumed to be independent and identically distributed as Normal(0, σ²), the OLS estimator of β is identical to the MLE. OLS does not require the normality assumption to obtain the point estimates; it only requires it for exact distributional results in small samples. MLE for non-normal error distributions (e.g. logistic, Poisson) yields different estimating equations and requires iterative optimization rather than the OLS closed form.

OLS treats all observations equally. WLS generalizes this by assigning a weight wᵢ to each observation, minimizing Σwᵢ(yᵢ − xᵢᵀβ)². WLS is the appropriate estimator when the error variances are known to differ proportionally across observations (heteroskedasticity with a known structure), or when observations represent aggregates with different reliabilities. Under the Gauss-Markov framework, WLS is more efficient than OLS in the presence of heteroskedasticity.