Regression and Smoothing Splines
Also known as: splines, cubic splines, natural splines, smoothing splines, B-spline regression, regresyon spline'ları
Regression splines model a nonlinear relationship by fitting piecewise polynomials that join smoothly at a set of points called knots. Cubic and natural splines are the most common, and smoothing splines add a roughness penalty that automatically balances fit against smoothness. Splines are the standard flexible building block for univariate nonlinear regression and the basis of generalized additive models.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use regression or smoothing splines whenever you need to model a smooth, nonlinear relationship between a response and a single continuous predictor flexibly and stably — as a standalone curve fit, as the smooth terms inside a generalized additive model, or to capture nonlinear trends and seasonal shapes in time series. Penalized smoothing splines are attractive because they sidestep manual knot placement, requiring only a smoothing parameter that can be chosen automatically. Splines extrapolate poorly beyond the data (natural splines partly mitigate edge behaviour), and for multivariate nonlinear surfaces with interactions, tensor-product splines or other methods are needed. When the relationship is plausibly linear, ordinary regression is simpler.
Strengths & limitations
- Flexible, smooth fits without the instability of high-degree global polynomials.
- Reduce to linear regression on a basis, so estimation and inference are standard.
- Smoothing splines avoid manual knot selection via a single penalty parameter.
- Form the interpretable smooth components of generalized additive models.
- Extrapolate unreliably beyond the range of the observed data.
- Unpenalized splines are sensitive to the number and placement of knots.
- Primarily univariate; multivariate surfaces with interactions need tensor products or other methods.
- Choice of basis, degree, and smoothing parameter all affect the result.
Frequently asked
What is a knot in a spline?
A knot is a point along the predictor's range where one polynomial piece ends and the next begins. The pieces are constrained to join smoothly there. More knots allow more flexibility; their number and placement (or, for smoothing splines, the penalty) control how wiggly the fitted curve can be.
What is the difference between a regression spline and a smoothing spline?
A regression spline uses a modest number of chosen knots and is fit by ordinary least squares on the spline basis. A smoothing spline places a knot at every data point and controls flexibility through a roughness penalty with parameter λ, so you tune one penalty instead of selecting knots.
Why use a natural cubic spline?
Ordinary cubic splines can behave erratically near the boundaries where data are sparse. A natural cubic spline adds constraints that force the function to be linear beyond the outermost knots, reducing wild boundary behaviour and the variance of the fit at the edges.
Sources
- Eilers, P. H. C., & Marx, B. D. (1996). Flexible smoothing with B-splines and penalties. Statistical Science, 11(2), 89–121. DOI: 10.1214/ss/1038425655 ↗
- Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer. ISBN: 978-0-387-84857-0
How to cite this page
ScholarGate. (2026, June 2). Regression and Smoothing Splines. ScholarGate. https://scholargate.app/en/machine-learning/regression-splines
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Generalized Additive ModelMachine learning↔ compare
- LOESSMachine learning↔ compare
- MARSMachine learning↔ compare
- Polynomial RegressionStatistics↔ compare