Multivariate Regression
Multivariate regression models the joint dependence of several response variables on a set of predictors, accounting for the correlations among the responses.
Definition
Multivariate regression is the modeling of two or more response variables simultaneously as functions of common predictors, with inference accounting for the covariance structure among the responses.
Scope
This area covers the multivariate general linear model and its specializations: multivariate multiple regression with several continuous responses, multivariate analysis of variance for comparing group mean vectors, and dimension-reducing predictive methods such as partial least squares for many correlated predictors. It addresses estimation, hypothesis testing via multivariate test statistics, and the modeling of response covariance.
Sub-topics
Core questions
- How are several correlated responses modeled jointly on common predictors?
- How are multivariate hypotheses about regression and group means tested?
- How does modeling responses jointly differ from separate univariate regressions?
- How are many correlated predictors handled predictively?
Key theories
- Multivariate general linear model
- The multivariate linear model expresses a matrix of responses as a common design matrix times a coefficient matrix plus correlated errors, unifying multivariate regression and multivariate analysis of variance under one framework.
- Multivariate test statistics
- Hypotheses about coefficient or mean-vector matrices are tested with statistics such as Wilks's lambda and the Pillai, Hotelling-Lawley, and Roy criteria, which combine information across responses.
Clinical relevance
Multivariate regression is used when outcomes are inherently multidimensional, such as multiple correlated measurements per subject, and to compare groups across several outcomes simultaneously while controlling overall error rates.
History
The multivariate linear model and its associated test statistics were developed in the classical theory of multivariate analysis in the first half of the twentieth century. Later, methods such as partial least squares extended regression with many responses or predictors to high-dimensional and collinear settings common in chemometrics.
Debates
- Choice among multivariate test statistics
- Wilks's lambda, Pillai's trace, the Hotelling-Lawley trace, and Roy's largest root can give different conclusions and differ in power and robustness, so the choice among them is not always clear.
Key figures
- T. W. Anderson
- Harold Hotelling
- Samuel Wilks
Related topics
Seminal works
- anderson2003
- johnson2007
- mardia1979
Frequently asked questions
- Why model multiple responses jointly instead of one at a time?
- Joint modeling accounts for correlations among responses, allows tests of hypotheses across all responses simultaneously, and controls the overall error rate that separate univariate analyses would inflate.
- How does multivariate regression relate to MANOVA?
- Both are special cases of the multivariate general linear model; MANOVA is the version in which the predictors encode group membership and the focus is on comparing mean vectors across groups.