Which method should I use?

Recommendations for: predict a continuous numeric outcome from several predictor variables

1. Multivariate RegressionStatistics

   Multivariate regression is a linear regression method that predicts several continuous dependent variables at the same time from a shared set of predictors. As developed in standard treatments such as Johnson and Wichern's Applied Multivariate Statistical Analysis (2007), each response equation can be fitted by ordinary least squares while the covariance structure of the residuals is used for joint testing across outcomes.

2. Bayesian Multiple linear regressionStatistics

   Bayesian Multiple Linear Regression models a continuous outcome as a linear combination of several predictors, but instead of producing a single point estimate it yields a full posterior distribution over all regression coefficients and the error variance. This makes uncertainty quantification explicit and allows seamlessly incorporating prior knowledge from theory or previous studies.

3. Robust Multiple linear regressionStatistics

   Robust multiple linear regression estimates the linear relationship between a continuous outcome and several predictors while being resistant to outliers and violations of the normality assumption. Instead of minimising the sum of squared residuals, it uses a bounded loss function — most commonly Huber's or Tukey's bisquare — so that extreme observations receive limited influence on the estimated coefficients.

4. Robust Discriminant AnalysisStatistics

   Robust Discriminant Analysis is a classification method that separates groups with a linear discriminant function while resisting the influence of outliers. It replaces the classical mean and covariance with a high-breakdown estimator such as the Minimum Covariance Determinant (MCD), an approach developed by Hawkins & McLachlan (1997) and Croux & Dehon (2001).

5. Linear Regression (ML)Machine Learning

   Linear regression fits a straight-line relationship between one or more input features and a continuous numeric outcome by minimising the sum of squared prediction errors. As a machine-learning model it is trained on labeled examples and evaluated on held-out data, making it the simplest supervised learning baseline for any regression task.

6. Regularized linear regressionMachine Learning

   Regularized linear regression adds a penalty term to the ordinary least-squares objective, shrinking or zeroing out coefficients to reduce overfitting and handle multicollinearity. The three main variants — Ridge (L2 penalty), Lasso (L1 penalty), and Elastic Net (combined L1+L2) — make linear regression usable even when features outnumber observations or predictors are highly correlated.