Particle Filter (Sequential Monte Carlo)

The Kalman filter is the exact Bayesian solution for linear-Gaussian state-space models, representing the posterior as a Gaussian characterised by a mean and covariance. It is computationally cheap but breaks down when the model is nonlinear or the noise is non-Gaussian. The particle filter makes no linearity or Gaussianity assumption: it represents the posterior non-parametrically as a cloud of weighted samples and can handle arbitrary model forms. The trade-off is computational cost — particles must be propagated individually and N may need to be large.

Weight degeneracy occurs when, after a few time steps, essentially all of the total weight concentrates on one or two particles, so the effective sample size N_eff collapses toward one. The standard remedy is resampling: draw N new particles with replacement from the current set in proportion to their weights and reset all weights to 1/N. This discards low-weight particles and duplicates high-weight ones, restoring diversity. Resampling is triggered adaptively when N_eff falls below a user-set threshold (commonly N/2). Care must be taken because resampling introduces its own problem — sample impoverishment — so MCMC rejuvenation or kernel-based regularisation can be added.

There is no universal answer. The required N depends on the state dimension, the peakedness of the likelihood, and the required accuracy. A common diagnostic is to run the filter with increasing N (e.g., 100, 500, 1000, 5000) and check that posterior summaries (means, credible intervals) stabilise. For low-dimensional tracking problems (2–4 dimensions) N in the hundreds to low thousands is often sufficient; for higher dimensions the requirement grows rapidly, and structure-exploiting methods (Rao-Blackwellisation, flow-based proposals) become necessary.

Static parameters do not change over time, so naive particle filtering on the joint state-parameter space collapses because the parameter particles never move. Practical solutions include: artificially jittering parameters (kernel smoothing / Liu-West filter), particle Markov chain Monte Carlo (PMCMC) which uses the particle filter as a likelihood estimator inside an MCMC scheme for the parameters, and sequential Monte Carlo squared (SMC2). PMCMC, introduced by Andrieu, Doucet, and Holenstein (2010, Journal of the Royal Statistical Society B), is the most principled and widely used approach.