Factor Analysis

Exploratory factor analysis (EFA) discovers latent structure from data without prior theory; you let the data guide how many factors and which variables load on which factors. Use EFA when investigating a new construct or scale. Confirmatory factor analysis (CFA) tests a hypothesized factor structure specified before analysis based on theory and prior research. CFA provides formal fit tests and is more statistically rigorous. Standard practice: conduct EFA on training sample to propose model, then validate with CFA on independent test sample (or use cross-validation). Never conduct EFA on the same sample used for CFA validation.

Multiple criteria guide factor extraction: (1) Kaiser criterion (eigenvalue > 1) is traditional but often suggests too many factors. (2) Scree plot: eigenvalues plotted in decreasing order; retain factors before the elbow (point where diminishing returns begin). The 'scree' is the slope after the elbow. (3) Parallel analysis: compare empirical eigenvalues to those from random permutations; retain only factors exceeding random baseline. Parallel analysis is most statistically defensible. (4) Interpretability: retain only factors you can conceptually label and explain. Different criteria often yield different numbers; consult multiple methods and theory.

A factor loading of 0.6 is the correlation between an observed variable and a latent factor; it indicates the variable contributes to measuring that factor. The squared loading (0.6² = 0.36) is the proportion of variable variance explained by the factor. Loadings > 0.4 are minimally acceptable; > 0.6 is strong. A reliability coefficient (Cronbach's alpha, typically ≥0.70) measures internal consistency: the degree to which multiple variables measuring the same construct correlate with each other. Loading shows relation to the latent factor; alpha shows whether variables together consistently measure one construct. Both matter: high loadings and high alpha together support a homogeneous latent dimension.

Communality is the proportion of a variable's variance explained by all factors (sum of squared loadings across factors). Low communalities (<0.3) suggest the variable is poorly represented by the factors, either because: (1) it measures a unique construct not captured by the factor model, (2) it is unreliable/noisy, or (3) the factor solution is incorrect. Options: remove the variable (simplify model if theoretically justified), add factors (may overfit), or investigate whether the variable genuinely belongs in the scale. Always examine item content; sometimes low communality reflects a poorly worded item rather than conceptual misfit.