How is classical MDS related to PCA?

When dissimilarities are Euclidean distances computed from data, classical scaling yields the same low-dimensional configuration as principal component analysis of the centered data.

What is stress in multidimensional scaling?

Stress is a normalized measure of the discrepancy between the fitted inter-point distances and the target dissimilarities; lower stress indicates a better-fitting configuration.

Multidimensional Scaling

Multidimensional scaling places objects in a low-dimensional space so that the inter-object distances reproduce a given matrix of dissimilarities as closely as possible.

Definition

Multidimensional scaling is a family of methods that embed objects as points in a low-dimensional space so that the distances between points approximate observed pairwise dissimilarities under a chosen loss criterion.

Scope

This topic covers classical (metric) scaling, in which a configuration is recovered exactly from Euclidean distances via an eigen-decomposition of a doubly centered distance matrix, and nonmetric scaling, which preserves only the rank order of dissimilarities by minimizing a stress criterion. It addresses the relationship to principal coordinates analysis and the assessment of fit.

Core questions

Given only pairwise dissimilarities, how can objects be positioned in a low-dimensional space?
When can a configuration be recovered exactly, and when must fit be optimized iteratively?
How is the quality of a scaling solution measured?
How does metric scaling relate to principal component and principal coordinates analysis?

Key theories

Classical (metric) scaling: When dissimilarities are Euclidean distances, double-centering the squared-distance matrix yields a positive semidefinite matrix whose leading eigenvectors give the coordinates, recovering the configuration up to rotation and translation.
Nonmetric scaling and stress minimization: When only the ordering of dissimilarities is meaningful, a monotone transformation and an iterative minimization of a stress function fit a configuration whose distances are monotonically related to the dissimilarities.

Clinical relevance

Multidimensional scaling is used to visualize similarity data such as perceptual judgments, genetic or geographic distances, and document or network proximities, turning a dissimilarity matrix into an interpretable map.

History

Metric scaling was formalized in the mid-twentieth century and connected to principal coordinates by Gower, while Kruskal and Shepard introduced nonmetric scaling based on monotone stress minimization, broadening the method to ordinal dissimilarity data.

Key figures

Warren Torgerson
Joseph Kruskal
John Gower

Seminal works

mardia1979
coxcox2001
borg2005

Frequently asked questions

How is classical MDS related to PCA?: When dissimilarities are Euclidean distances computed from data, classical scaling yields the same low-dimensional configuration as principal component analysis of the centered data.
What is stress in multidimensional scaling?: Stress is a normalized measure of the discrepancy between the fitted inter-point distances and the target dissimilarities; lower stress indicates a better-fitting configuration.