Time Series Variational Inference
Variational Inference for Time Series Models · Also known as: time-series VI, variational Bayes for time series, TSVI, sequential variational inference
Time series variational inference applies variational Bayes to sequential data, approximating the intractable posterior over latent states and parameters with a tractable family of distributions. By maximising the evidence lower bound (ELBO), it delivers fast, scalable Bayesian inference for state-space models, dynamic latent variable models, and other time-ordered probabilistic systems.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use time series variational inference when you have a probabilistic generative model for sequential data and MCMC is too slow — for example with long time series, large datasets, or high-dimensional latent spaces. It is especially effective when online or streaming inference is needed or when the model is embedded in a deep learning pipeline (variational autoencoders, neural SSMs). Prefer MCMC methods such as sequential Monte Carlo or particle MCMC when exact posterior coverage matters more than speed, when the dataset is small enough for MCMC to be tractable, or when the variational family may be too restrictive to capture multimodal or strongly non-Gaussian posteriors.
Strengths & limitations
- Scales to long sequences and large datasets where MCMC is computationally prohibitive.
- Provides a deterministic, gradient-based optimisation objective (ELBO) that is straightforward to monitor.
- Integrates naturally with deep learning: inference networks and generative networks can be jointly trained end to end.
- Delivers analytic or sample-based uncertainty estimates over hidden states and forecasts.
- Amortised variants generalise across datasets, enabling fast inference on held-out sequences.
- The variational posterior is only an approximation; the ELBO can be loose when the chosen family is misspecified.
- Mean-field variants underestimate posterior variance and fail to capture temporal correlations between latent states.
- Convergence to a local optimum of the ELBO is not guaranteed; results depend on initialisation and learning-rate choices.
- Model comparison via ELBO is a lower bound on marginal likelihood and may not rank models correctly.
Frequently asked
How is time series variational inference different from the Kalman filter?
The Kalman filter computes the exact posterior for linear Gaussian state-space models. Variational inference is an approximation applicable to nonlinear and non-Gaussian models where the Kalman filter's exactness no longer holds. In the linear Gaussian case, a structured variational family recovers the Kalman smoother as a special case.
What is the ELBO and why is it maximised instead of the marginal likelihood?
The evidence lower bound (ELBO) is a tractable surrogate for the intractable log marginal likelihood log p(y). Maximising the ELBO is equivalent to minimising the KL divergence between the variational distribution and the true posterior. The gap between the ELBO and the true log-evidence equals this KL divergence and is always non-negative.
Can variational inference give wrong uncertainty estimates?
Yes. Because VI minimises forward KL divergence (or maximises ELBO, which is equivalent to reverse KL minimisation in the standard formulation), the variational posterior tends to underestimate variance and may collapse onto one mode of a multimodal posterior. Always validate with posterior predictive checks and, where feasible, compare against an MCMC benchmark on a small dataset.
When should I prefer sequential Monte Carlo over variational inference for time series?
Sequential Monte Carlo (particle filters/smoothers) provides asymptotically exact posterior samples and handles highly nonlinear or non-Gaussian dynamics without approximation bias. Prefer SMC when the dataset is small to moderate, when exact uncertainty quantification is critical, or when the variational family cannot be made flexible enough. Use VI when scalability and speed are paramount.
What software implements time series variational inference?
Pyro and NumPyro (probabilistic programming in PyTorch / JAX) support structured variational inference for user-defined generative models including state-space models. TensorFlow Probability offers similar capabilities. Stan supports variational Bayes (meanfield and fullrank) for any model specified in its modelling language.
Sources
- Blei, D. M., Kucukelbir, A. & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518), 859-877. DOI: 10.1080/01621459.2017.1285773 ↗
- Jordan, M. I., Ghahramani, Z., Jaakkola, T. S. & Saul, L. K. (1999). An introduction to variational methods for graphical models. Machine Learning, 37(2), 183-233. DOI: 10.1023/A:1007665907178 ↗
How to cite this page
ScholarGate. (2026, June 3). Variational Inference for Time Series Models. ScholarGate. https://scholargate.app/en/bayesian/time-series-variational-inference
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Dynamic Variational InferenceBayesian↔ compare
- Kalman FilterBayesian↔ compare
- Sequential Monte CarloBayesian↔ compare
- Time series Bayesian inferenceBayesian↔ compare
- Time series MCMCBayesian↔ compare
- Variational InferenceBayesian↔ compare