Lucas-Kanade Optical Flow

The brightness constancy assumption states that a pixel's intensity does not change as it moves between frames: I(x, y, t) = I(x+u, y+v, t+1). This assumption fails when illumination changes (shadows, reflections), or when pixels are occluded and revealed (exposing different underlying intensity). For robust tracking, preprocessing (histogram equalization) or robust estimation methods can help mitigate failures.

The aperture problem occurs when local feature structure is ambiguous: pixels along a straight edge can move in multiple directions without changing their local appearance. Lucas-Kanade addresses this by using a neighborhood window: the combined gradients from all pixels in the window provide enough information to uniquely determine the motion, assuming the window contains varied texture.

Pyramidal implementation builds an image pyramid (coarse-to-fine), computing optical flow at coarse levels first (where motion appears small), then refining at finer levels. This coarse-to-fine approach handles large motions that would violate the small-motion assumption at a single scale. Modern multi-scale trackers extend this idea to multiple resolution levels.

Lucas-Kanade estimates motion only at detected feature points (corners, edges), not at every pixel. This sparsity makes computation efficient and interpretable: you get motion vectors at interest points. Dense optical flow methods (Horn-Schunck, modern learning-based methods) estimate motion at every pixel, providing complete flow fields but at higher computational cost.