Feature Detection and Description

Feature detection and description find distinctive local points in an image and summarize their surrounding appearance so that the same physical points can be recognized and matched across different images.

Definition

A feature is a distinctive image location with an associated descriptor vector; detection locates such points repeatably, and description encodes their appearance for comparison.

Scope

This topic covers corner and blob detectors such as the Harris detector, scale-invariant keypoint detection, local descriptors that encode the neighborhood of a keypoint, and the invariance to scale, rotation, and illumination that makes features reliable for matching.

Core questions

• Which image locations are distinctive and repeatable enough to match?
• How is the local appearance around a point encoded compactly?
• How are descriptors made invariant to scale, rotation, and lighting?
• How are features matched between images?

Key concepts

• Corner and blob detection
• Structure tensor
• Scale-space extrema
• Local descriptors
• Invariance to scale and rotation
• Feature matching

Key theories

Corner detection

Corners are located where image intensity varies strongly in all directions, identified from the eigenvalues of the local gradient structure tensor, giving points that are well localized and stable under small viewpoint changes.

Scale-invariant feature transform

SIFT detects keypoints as extrema in a difference-of-Gaussian scale space and describes each by a histogram of gradient orientations, producing descriptors robust to scale, rotation, and moderate illumination and viewpoint change.

Clinical relevance

Local features are the workhorse of image matching, panorama stitching, structure-from-motion and visual localization, object instance recognition, and augmented-reality tracking.

History

The Harris detector of 1988 gave a robust corner measure, and Lowe's SIFT in 2004 made scale- and rotation-invariant matching practical, dominating wide-baseline matching until learned features and deep networks emerged.

Key figures

• Chris Harris
• David Lowe

Seminal works

• harris1988
• lowe2004

Frequently asked questions

Why are corners good features but flat regions are not?

A corner looks different in every direction, so its position can be pinned down precisely and matched unambiguously, whereas a flat or uniformly edged region looks the same when shifted, making it ambiguous to match.

Why does a descriptor need to be invariant?

The same scene point appears at different scales, rotations, and brightness across photos; a descriptor that stays nearly constant under those changes lets the point be recognized as the same in different images.