Sequential Monte Carlo with Missing Data
Also known as: SMC with missing data, particle filter with missing observations, SMC missing observations, particle smoothing with incomplete data
Sequential Monte Carlo (SMC) with missing data extends the standard particle filter to state-space models in which some observations are absent. When an observation is missing at a given time step the update step is simply skipped: particles are propagated forward through the transition model without reweighting, preserving exact Bayesian inference under any missing-data pattern as long as missingness is ignorable (missing at random or missing completely at random).
Key highlights
- Handles nonlinear and non-Gaussian state-space models with no Gaussianity assumption.
- Missing steps are treated exactly without imputation: the predict step alone propagates the posterior.
- Uncertainty inflates naturally across gaps, giving honest credible intervals at every time point.
- Scales to multivariate states and complex emission models through the particle representation.
- Can be combined with SMC samplers or MCMC kernels for static parameter estimation simultaneously.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use SMC with missing data when you have sequential or time-series observations from a nonlinear or non-Gaussian state-space model and some observations are intermittently absent. It is ideal for ecological tracking, sensor fusion, financial volatility estimation, and epidemiological surveillance where dropouts are common. Prefer it over imputation-then-filter pipelines when the missingness pattern is irregular or when propagating uncertainty through gaps matters for downstream decisions. Do not use it when the state-space model is linear-Gaussian and data are dense — a Kalman filter with missing-step handling is faster and exact. Avoid if missingness is informative (not-at-random) without explicitly modelling the missing-data mechanism.
Strengths & limitations
- Handles nonlinear and non-Gaussian state-space models with no Gaussianity assumption.
- Missing steps are treated exactly without imputation: the predict step alone propagates the posterior.
- Uncertainty inflates naturally across gaps, giving honest credible intervals at every time point.
- Scales to multivariate states and complex emission models through the particle representation.
- Can be combined with SMC samplers or MCMC kernels for static parameter estimation simultaneously.
- Particle degeneracy can occur when gaps are very long or the state space is high-dimensional, requiring large particle counts.
- Computational cost grows linearly with both the number of time steps and the number of particles.
- Valid only under the missing-at-random or missing-completely-at-random assumption; informative missingness requires a joint model.
- Smoothed estimates (backward pass) are more expensive than filtered estimates and require additional algorithms such as the forward-filter backward-smoother.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Do I need to impute missing observations before running SMC?
No. The correct approach is to skip the likelihood update at missing time steps and propagate particles forward using only the transition model. Imputing and then filtering introduces additional uncertainty not properly accounted for and can bias the posterior.
How many particles do I need when there are long gaps?
More than usual. During a gap particles spread according to the process noise without any corrective reweighting, so diversity must be maintained by a larger ensemble. A practical rule is to pilot-run with increasing N until posterior summaries stabilise, with 1000–5000 particles being a common starting range for moderate-dimensional problems.
Can SMC handle data that are missing not at random?
Not without modification. If the probability that y_t is missing depends on the unobserved y_t or x_t itself, the dropout mechanism must be modelled jointly with the state. This substantially complicates both model specification and inference.
What is the difference between filtering and smoothing with missing data?
Filtering gives p(x_t | y_{1:t}): the state distribution using only past and present data. Smoothing gives p(x_t | y_{1:T}): the state distribution using all data, including future observations. Smoothed estimates are more accurate but require a backward pass (e.g., the particle smoother) after the forward filter.
Is SMC with missing data the same as a Kalman filter with missing observations?
They implement the same conceptual idea — skip the update when data are absent — but for different model classes. The Kalman filter is restricted to linear-Gaussian models; SMC applies to any state-space model where the transition and emission densities can be evaluated or sampled.
Sources
- 1.Doucet, A., de Freitas, N., & Gordon, N. (Eds.) (2001). Sequential Monte Carlo Methods in Practice. Springer, New York.ISBN 978-0387951461
- 2.Chopin, N., & Papaspiliopoulos, O. (2020). An Introduction to Sequential Monte Carlo. Springer, Cham.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). Sequential Monte Carlo with Missing Data. ScholarGate. https://scholargate.app/bayesian/sequential-monte-carlo-with-missing-data