Adaptive LMS Filter
Also known as: LMS Filter, Adaptive LMS Algorithm, Gradient Descent Filtering
The Least Mean Squares (LMS) filter is an adaptive signal processing algorithm that continuously updates filter coefficients to minimize the squared error between the filter output and a desired signal. Introduced by Bernard Widrow and Marcian Hoff in 1960, the LMS algorithm is one of the most widely used adaptive filtering techniques due to its simplicity, low computational cost, and ability to track time-varying signals.
Key highlights
- Very simple algorithm requiring minimal computational resources—only vector products and additions per sample
- Robust convergence properties for a wide range of learning rate settings
- Proven effectiveness in many practical applications from telecommunications to biomedical engineering
- Naturally handles non-stationary signals that change over time through continuous adaptation
- Requires no knowledge of input signal statistics or system parameters for implementation
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use the LMS algorithm for real-time adaptive filtering when the signal characteristics change over time or are initially unknown. It is ideal for applications like echo cancellation, channel equalization, and system identification. The simplicity and low computational cost make it suitable for embedded systems and real-time processing. Avoid LMS when very fast convergence is required—more sophisticated variants like RLS offer faster convergence at higher computational cost.
Strengths & limitations
- Very simple algorithm requiring minimal computational resources—only vector products and additions per sample
- Robust convergence properties for a wide range of learning rate settings
- Proven effectiveness in many practical applications from telecommunications to biomedical engineering
- Naturally handles non-stationary signals that change over time through continuous adaptation
- Requires no knowledge of input signal statistics or system parameters for implementation
- Convergence is slow, requiring many samples to reach optimal filter coefficients compared to batch methods
- Learning rate selection requires careful tuning—too large leads to instability, too small leads to slow convergence
- Poor performance in high-noise environments compared to optimal Wiener filtering
- Computational misadjustment error occurs after convergence due to noise in the gradient estimates
- No guarantee of optimal solution in nonlinear or time-varying multimodal error surfaces
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How do I choose the learning rate (step size) for LMS?
A good starting point is μ < 1/(10·P), where P is the input signal power or variance. For a given problem, start conservatively with a small μ and gradually increase until convergence becomes acceptable. Too large a μ causes oscillation and instability; too small a μ causes very slow convergence. Normalized LMS (NLMS) automatically adapts μ based on input power for more robust performance.
Why does LMS convergence slow down with correlated input signals?
When input samples are highly correlated, the gradient estimates become noisy and variable across different frequency components, leading to slow and uneven convergence. The eigenvalue spread of the input autocorrelation matrix determines convergence speed. Preprocessing with whitening or using normalized variants like NLMS helps mitigate this issue.
What is the difference between LMS and NLMS (Normalized LMS)?
NLMS modifies the learning rate to be adaptive based on instantaneous input power: μ(n) = μ₀/(δ + ||x(n)||²), where δ is a small constant. This normalization makes convergence speed less dependent on input signal power and improves performance with highly correlated inputs. NLMS is slightly more computationally expensive but often worth it for improved robustness.
Can LMS be used for nonlinear filtering?
Standard LMS is inherently linear. However, nonlinear variants exist, such as LMS with nonlinear activation functions in each tap (artificial neural networks) or kernel-based adaptive filters that implicitly operate in high-dimensional feature spaces. These nonlinear extensions are more computationally expensive but can adapt to nonlinear relationships between input and desired output.
Sources
- 1.Widrow, B., & Hoff, M. E. (1960). Adaptive Switching Circuits. IRE Wescon Convention Record, 4, 96–104.
- 2.Haykin, S. (2002). Adaptive Filter Theory (4th ed.). Prentice Hall.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Adaptive LMS Filter. ScholarGate. https://scholargate.app/signal-processing/adaptive-lms-filter