Bayesian Factor Analysis

Classical EFA returns a single loading matrix estimated by maximum likelihood and selects the number of factors via heuristics such as scree plots or parallel analysis. Bayesian factor analysis places priors on the loadings and residual variances, samples the full posterior by MCMC, and selects the number of factors using marginal likelihood criteria (DIC, WAIC). The result is a distribution over loadings rather than a point estimate, which makes uncertainty explicit and enables regularisation of small samples.

In any factor model, multiplying the loading matrix by an orthogonal rotation matrix and the factor scores by its inverse leaves the likelihood unchanged. In classical EFA this is resolved by post-hoc rotation (Varimax, Oblimin). In the Bayesian setting, rotational indeterminacy is instead resolved by identification constraints imposed before sampling — for example, fixing the loading matrix to be lower triangular. Without such constraints the MCMC sampler encounters a non-identified surface and the chains do not converge meaningfully.

Within the Bayesian framework, the number of factors is chosen by fitting models with k, k+1, and k−1 factors and comparing them using marginal likelihood approximations such as DIC (Deviance Information Criterion) or WAIC (Widely Applicable Information Criterion). The model with the lower DIC or WAIC is preferred. This replaces the scree plot and parallel analysis heuristics used in classical EFA with a formally probabilistic criterion.

The minimum recommended sample is 30 observations. Below this threshold MCMC convergence is difficult to achieve and the posterior is shaped almost entirely by the prior, meaning the data contribute little. With 30 to 100 observations the prior exerts notable influence and a sensitivity analysis — re-running the model with alternative reasonable priors — should be reported. With larger samples the posterior concentrates near the maximum-likelihood solution and Bayesian and classical estimates tend to agree closely.