Regression and Smoothing Splines
Also known as: splines, cubic splines, natural splines, smoothing splines, B-spline regression, regresyon spline'ları
Regression splines model a nonlinear relationship by fitting piecewise polynomials that join smoothly at a set of points called knots. Cubic and natural splines are the most common, and smoothing splines add a roughness penalty that automatically balances fit against smoothness. Splines are the standard flexible building block for univariate nonlinear regression and the basis of generalized additive models.
Key highlights
- Flexible, smooth fits without the instability of high-degree global polynomials.
- Reduce to linear regression on a basis, so estimation and inference are standard.
- Smoothing splines avoid manual knot selection via a single penalty parameter.
- Form the interpretable smooth components of generalized additive models.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use regression or smoothing splines whenever you need to model a smooth, nonlinear relationship between a response and a single continuous predictor flexibly and stably — as a standalone curve fit, as the smooth terms inside a generalized additive model, or to capture nonlinear trends and seasonal shapes in time series. Penalized smoothing splines are attractive because they sidestep manual knot placement, requiring only a smoothing parameter that can be chosen automatically. Splines extrapolate poorly beyond the data (natural splines partly mitigate edge behaviour), and for multivariate nonlinear surfaces with interactions, tensor-product splines or other methods are needed. When the relationship is plausibly linear, ordinary regression is simpler.
Strengths & limitations
- Flexible, smooth fits without the instability of high-degree global polynomials.
- Reduce to linear regression on a basis, so estimation and inference are standard.
- Smoothing splines avoid manual knot selection via a single penalty parameter.
- Form the interpretable smooth components of generalized additive models.
- Extrapolate unreliably beyond the range of the observed data.
- Unpenalized splines are sensitive to the number and placement of knots.
- Primarily univariate; multivariate surfaces with interactions need tensor products or other methods.
- Choice of basis, degree, and smoothing parameter all affect the result.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is a knot in a spline?
A knot is a point along the predictor's range where one polynomial piece ends and the next begins. The pieces are constrained to join smoothly there. More knots allow more flexibility; their number and placement (or, for smoothing splines, the penalty) control how wiggly the fitted curve can be.
What is the difference between a regression spline and a smoothing spline?
A regression spline uses a modest number of chosen knots and is fit by ordinary least squares on the spline basis. A smoothing spline places a knot at every data point and controls flexibility through a roughness penalty with parameter λ, so you tune one penalty instead of selecting knots.
Why use a natural cubic spline?
Ordinary cubic splines can behave erratically near the boundaries where data are sparse. A natural cubic spline adds constraints that force the function to be linear beyond the outermost knots, reducing wild boundary behaviour and the variance of the fit at the edges.
Sources
- 1.Eilers, P. H. C., & Marx, B. D. (1996). Flexible smoothing with B-splines and penalties. Statistical Science, 11(2), 89–121.
- 2.Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.ISBN 978-0-387-84857-0
You have read it. What now?
Cite this page
ScholarGate. (2026, June 2). Regression Splines. ScholarGate. https://scholargate.app/machine-learning/regression-splines