Factor Analysis

Factor analysis models the correlations among observed variables as arising from a smaller number of unobserved common factors plus variable-specific uniqueness.

Definition

Factor analysis is a latent-variable model in which each observed variable is expressed as a linear combination of a few common factors and an independent specific error, so that the off-diagonal covariance structure is reproduced by the common factors alone.

Scope

This topic covers the common-factor model, estimation of loadings and uniquenesses by methods such as principal-factor and maximum likelihood, factor rotation for interpretability, the distinction between exploratory and confirmatory factor analysis, and factor-score estimation. It also addresses identification and the indeterminacy of the factor solution.

Core questions

• How many common factors underlie a set of correlated measurements?
• How are factor loadings estimated and how is the solution rotated to be interpretable?
• How does the common-factor model differ from a principal-component decomposition?
• When is a factor model identified, and how should factor scores be obtained?

Key theories

Common-factor decomposition

The covariance matrix is modeled as the sum of a low-rank common part, generated by shared factors, and a diagonal uniqueness part, separating shared variance from variable-specific variance.

Rotational indeterminacy

Because any orthogonal rotation of the factors reproduces the same covariance structure, the factor solution is determined only up to rotation, motivating rotation criteria such as varimax to aid interpretation.

Clinical relevance

Factor analysis is central to psychometrics and survey research for constructing and validating scales, and is used across the social and biological sciences to identify latent dimensions underlying many measured indicators.

History

Factor analysis grew out of Spearman's early-twentieth-century work on a general factor of intelligence and was extended by Thurstone into multiple-factor analysis with rotation. Maximum-likelihood estimation and confirmatory models were later formalized, embedding factor analysis within the broader theory of latent-variable and structural-equation modeling.

Debates

Factor analysis versus principal components

The two methods are often confused; factor analysis posits an explicit error model and targets common variance, whereas principal components analyze total variance with no error term, and they can give materially different solutions.

Key figures

• Charles Spearman
• L. L. Thurstone
• Harry Harman

Seminal works

• mardia1979
• harman1976
• anderson2003

Frequently asked questions

What is the difference between exploratory and confirmatory factor analysis?

Exploratory factor analysis estimates the number and pattern of factors from the data, while confirmatory factor analysis tests a pre-specified factor structure with constraints on which variables load on which factors.

Why are factors rotated?

Rotation exploits the indeterminacy of the solution to find a loading pattern that is easier to interpret, typically one in which each variable loads strongly on few factors.