Multivariate Multiple Regression

Multivariate multiple regression predicts several continuous responses simultaneously from a common set of predictors, modeling the correlations among the responses.

Definition

Multivariate multiple regression is a linear model in which a vector of responses is regressed on shared predictors with correlated errors, estimated and tested while accounting for the response covariance structure.

Scope

This topic covers the multivariate linear model with a matrix of responses, least-squares estimation that coincides with equation-by-equation regression for point estimates, the role of the error covariance in inference, multivariate tests of regression hypotheses, and prediction regions for the vector of responses.

Core questions

• How are several continuous responses predicted jointly from common predictors?
• When does joint estimation differ from separate univariate regressions?
• How are hypotheses about the coefficient matrix tested?
• How are joint prediction regions for the responses constructed?

Key theories

Matrix least squares

The least-squares estimate of the coefficient matrix is obtained column by column, so point estimates equal those of separate univariate regressions, while the estimated error covariance ties the responses together for inference.

Multivariate inference on coefficients

Tests of hypotheses about the coefficient matrix use multivariate statistics derived from the error and hypothesis sum-of-squares-and-cross-products matrices, combining evidence across responses.

Clinical relevance

Multivariate multiple regression is used when several outcomes are measured together and share predictors, enabling joint hypothesis tests and prediction regions that respect the correlations among outcomes.

History

The multivariate linear model and its inferential theory were developed within the classical multivariate analysis of the early to mid-twentieth century, generalizing univariate regression to vector responses and supplying the multivariate test statistics still in use.

Key figures

• T. W. Anderson
• Samuel Wilks

Seminal works

• anderson2003
• johnson2007
• mardia1979

Frequently asked questions

If point estimates equal separate regressions, why use the multivariate model?

Because joint inference, hypothesis testing across responses, and prediction regions depend on the response covariance, which the multivariate model captures but separate univariate regressions ignore.

What is the role of the error covariance matrix?

It quantifies how the response errors covary and enters the multivariate test statistics and prediction regions, accounting for the dependence among the responses.