Principal Component Analysis
Principal Component Analysis (PCA) · Also known as: Temel Bileşenler Analizi (PCA), PCA, principal components analysis, Karhunen-Loève transform
Principal Component Analysis (PCA) is an unsupervised dimensionality-reduction method — given its modern textbook treatment by Ian Jolliffe (2002) — that compresses high-dimensional data into fewer dimensions while preserving the maximum possible variance. It re-expresses correlated variables as a small set of uncorrelated principal components ordered by how much of the data's variation each one captures.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
+16 more
When to use it
Use PCA to explore or describe high-dimensional data with many continuous (or count) features that are correlated, when you want to reduce dimensions, remove redundancy, or visualize structure rather than predict a labelled outcome. It assumes continuous variables, genuine correlation among them, and standardized features, and works best when the number of observations comfortably exceeds the number of variables (at least about 30 observations). When the sample is small or variables approach observations in number, variable-selection approaches such as Lasso are safer.
Strengths & limitations
- Compresses many correlated variables into a few uncorrelated components while retaining maximum variance.
- Removes redundancy and multicollinearity, producing components that are orthogonal by construction.
- Assumption-light on distribution: no normality requirement, suitable for exploratory and descriptive work.
- Components ordered by explained variance make dimension selection transparent via scree plots and eigenvalues.
- Enables low-dimensional visualization (for example biplots) of otherwise high-dimensional data.
- On small samples (n below about 30) the covariance matrix is unstable and component loadings become unreliable.
- When the number of variables approaches the number of observations, PCA overfits and variable selection is preferable.
- Requires standardized, correlated, continuous features; with near-zero correlation the method is meaningless.
- Components are linear combinations of all variables and can be hard to interpret substantively.
Frequently asked
How many components should I keep?
Common rules are inspecting a scree plot for the elbow, keeping components with eigenvalues over one (the Kaiser criterion), or retaining enough components to reach about 80% cumulative explained variance. The right choice depends on how much variance you are willing to trade for simplicity.
Do I have to standardize my data first?
Yes. PCA is driven by variance, so without standardization a variable measured in large units would dominate the components purely because of its scale. Standardizing every feature first is a mandatory step here.
When is PCA inappropriate?
PCA is meaningless when the variables are nearly uncorrelated, because there is no redundant structure to compress. It is also unreliable on small samples (n below about 30), where the covariance matrix is unstable, and it overfits when the number of variables approaches the number of observations.
Is PCA the same as factor analysis?
They are related but distinct. PCA seeks orthogonal directions that maximize retained variance and is primarily a data-reduction tool, whereas factor analysis posits latent factors that explain shared variance. In some software PCA appears as an extraction option inside a factor-analysis menu.
Sources
- Jolliffe, I.T. (2002). Principal Component Analysis (2nd ed.). Springer. DOI: 10.1007/b98835 ↗
How to cite this page
ScholarGate. (2026, June 1). Principal Component Analysis (PCA). ScholarGate. https://scholargate.app/en/machine-learning/pca
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Factor AnalysisResearch Statistics↔ compare
- Hierarchical ClusteringMachine learning↔ compare
- Lasso RegressionMachine learning↔ compare