Time-MoE: Mixture-of-Experts Time-Series Foundation Model
Also known as: Time Mixture-of-Experts, Time-MoE Foundation Model, Sparse Time-Series Transformer, Zaman Karışık Uzmanlar Modeli
Time-MoE is a billion-scale autoregressive foundation model for universal time-series forecasting, introduced by Shi et al. in 2024 and accepted at ICLR 2025. It combines a decoder-only transformer architecture with sparse Mixture-of-Experts (MoE) feed-forward layers, enabling the model to scale to billions of parameters while activating only a small subset of expert networks per token—dramatically increasing capacity without proportional compute cost.
Key highlights
- Scales to billions of parameters with sublinear compute growth via sparse expert activation
- Strong zero-shot generalization across diverse time-series domains due to large-scale pretraining
- Multi-resolution forecasting heads allow simultaneous short- and long-horizon predictions from a single model
- Decoder-only autoregressive design enables flexible context length without architectural changes
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Time-MoE is appropriate when you need a general-purpose forecasting foundation model that can generalize across domains without task-specific training. It is particularly well-suited for zero-shot or few-shot forecasting on univariate series, large-scale deployments where inference efficiency matters, and settings where labeled data is scarce. Assumptions include well-behaved scalar time series; it is not designed for multivariate cross-series dependency modeling out of the box. Alternatives include TimesFM, Chronos, and Moirai for similar zero-shot settings.
Strengths & limitations
- Scales to billions of parameters with sublinear compute growth via sparse expert activation
- Strong zero-shot generalization across diverse time-series domains due to large-scale pretraining
- Multi-resolution forecasting heads allow simultaneous short- and long-horizon predictions from a single model
- Decoder-only autoregressive design enables flexible context length without architectural changes
- Primarily designed for univariate forecasting; multivariate cross-series relationships require adaptation
- Large model size demands substantial memory and infrastructure for deployment
- Autoregressive decoding accumulates prediction errors over long horizons
- Pretraining data distribution may not cover highly specialized or rare time-series domains
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How does Time-MoE differ from a standard transformer forecasting model?
Standard transformer forecasters use dense feed-forward layers where every parameter is activated for every token. Time-MoE replaces these with sparse MoE blocks, activating only a small subset of expert networks per token. This allows the total parameter count—and thus model capacity—to scale dramatically while keeping per-token compute roughly constant, enabling billion-parameter models at manageable inference cost.
Can Time-MoE handle multivariate time series?
Time-MoE is primarily architected for univariate forecasting, processing each series independently. While it can be applied channel-independently to multivariate data (treating each variable as a separate series), it does not natively model cross-variable dependencies. Tasks requiring explicit inter-series correlation modeling may benefit from purpose-built multivariate models instead.
What is Time-300B and why does it matter?
Time-300B is the large-scale pretraining corpus introduced alongside Time-MoE, comprising hundreds of billions of time-series observations spanning multiple domains including energy, finance, weather, and transportation. Pretraining on this diverse corpus enables Time-MoE to develop general temporal representations that transfer zero-shot to unseen datasets, similar to how web-scale text corpora underpin large language model generalization.
Sources
- 1.Shi, X., Wang, S., Nie, Y., Li, D., Ye, Z., Wen, Q., & Jin, M. (2024). Time-MoE: Billion-scale time series foundation models with mixture of experts. ICLR 2025.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 2). Time-MoE. ScholarGate. https://scholargate.app/deep-learning/time-moe