Unsupervised Learning

Unsupervised learning discovers structure in unlabeled data, finding groupings, low-dimensional representations, and latent factors without target outputs to imitate.

Definition

Unsupervised learning is the inference of structure from inputs alone, with no associated target values; algorithms seek compact descriptions of the data such as cluster assignments, low-dimensional coordinates, or generative latent variables that explain how the observed data could have arisen.

Scope

This area covers learning from data without labels: clustering into groups, dimensionality reduction and manifold learning, latent-variable and mixture models fit by the expectation-maximization algorithm, density estimation, and modern self-supervised and representation learning that creates training signals from the data itself.

Sub-topics

• clustering algorithms
• dimensionality reduction
• latent variable and mixture models
• self supervised and representation learning

Core questions

• What structure can be recovered from data without any labels?
• How are natural groupings or clusters defined and discovered?
• How can high-dimensional data be summarized by few coordinates?
• How do latent-variable models explain observations through hidden causes?

Key theories

Latent-variable models and EM

Many unsupervised models posit hidden variables that generate the data, and the expectation-maximization algorithm fits them by alternating between inferring the latent variables and updating parameters to increase likelihood.

Dimensionality reduction

Methods such as principal component analysis and manifold learning find low-dimensional representations that preserve the most important variation, enabling visualization, compression, and noise reduction.

Clustering structure

Clustering partitions data into groups of similar items, formalized variously through within-cluster distance, probabilistic mixtures, or density, with no single definition of the right number or shape of clusters.

Clinical relevance

Unsupervised learning is essential where labels are scarce or absent, supporting customer segmentation, anomaly detection, exploratory data analysis, and the pretraining of representations that power modern supervised and language systems; because there is no ground-truth target, evaluating unsupervised results is itself a subtle and important problem.

History

Unsupervised learning has roots in clustering and factor analysis from statistics and in self-organizing neural networks. The expectation-maximization algorithm, formalized in 1977, unified the fitting of latent-variable models, and in recent years self-supervised representation learning has become a dominant paradigm for pretraining large models on unlabeled data.

Debates

How to evaluate unsupervised results

Without labels there is no single correct answer, so judging clusterings or learned representations relies on indirect criteria, downstream task performance, or human interpretation, and different validity measures can disagree.

Key figures

• Arthur Dempster
• Donald Rubin
• Geoffrey Hinton
• Christopher Bishop

Seminal works

• bishop2006
• hastie2009
• dempster1977

Frequently asked questions

How can a model learn anything without labels?

Unsupervised methods exploit structure already present in the data, such as which points are close together, which directions carry the most variation, or which latent factors could have generated the observations. The data's own regularities provide the signal.

Why is unsupervised learning hard to evaluate?

There is no ground-truth target to compare against, so success is judged indirectly, for example by how interpretable the clusters are or how well a learned representation helps a later supervised task. Different criteria can rank the same result differently.