iTransformer: Inverted Transformer for Multivariate Time-Series Forecasting
Also known as: Inverted Transformer, iTransformer for Time Series, Inverted Attention Transformer, Ters Transformer
iTransformer is a deep-learning architecture for multivariate time-series forecasting introduced by Liu et al. at ICLR 2024. Its defining idea is to invert the conventional Transformer tokenisation strategy: instead of treating each time step as a token, iTransformer treats each variate (sensor channel or feature series) as a single token whose embedding encodes the full observed look-back window. Self-attention is then applied across variates to capture inter-series dependencies, while a feed-forward network within each token learns temporal patterns.
Key highlights
- Directly models inter-variate correlations through the attention mechanism without hand-crafted channel mixing
- Computationally efficient for high-variate datasets because attention sequence length scales with N (variates), not T (time steps)
- The feed-forward network retains the ability to capture variate-level temporal patterns despite the inverted token structure
- Simple, clean architectural modification to standard Transformer that is easy to implement and reproduce
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
iTransformer is well suited to multivariate forecasting tasks where cross-variable dependencies are informative and the number of variates is large relative to the horizon. It assumes the look-back window is long enough to provide meaningful per-variate embeddings and that inter-series correlations are stable across time. It is less appropriate for univariate forecasting (the inversion provides no advantage) or when variate counts are very small. Alternatives include PatchTST for patch-based temporal tokenisation, Crossformer for fine-grained cross-time-cross-variate attention, or N-BEATS and TimesNet for different inductive biases.
Strengths & limitations
- Directly models inter-variate correlations through the attention mechanism without hand-crafted channel mixing
- Computationally efficient for high-variate datasets because attention sequence length scales with N (variates), not T (time steps)
- The feed-forward network retains the ability to capture variate-level temporal patterns despite the inverted token structure
- Simple, clean architectural modification to standard Transformer that is easy to implement and reproduce
- Offers no direct advantage in purely univariate settings where cross-variate attention is vacuous
- Per-variate token embeddings conflate the entire look-back window into a single vector, potentially losing fine-grained temporal positional information
- Performance may degrade if inter-variate relationships are non-stationary or change over time
- Like most Transformer-based forecasters, requires substantial training data and GPU resources to outperform simpler linear baselines reliably
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What makes iTransformer different from a standard Transformer applied to time series?
A standard Transformer tokenises each time step, so attention captures temporal position dependencies. iTransformer tokenises each variate (channel), so attention captures inter-series dependencies instead. Temporal patterns are handled by the feed-forward sub-layer applied within each variate token. This architectural inversion is the sole but consequential change.
Is iTransformer suitable for univariate forecasting?
Not in its standard form. The inverted attention mechanism gains its benefit from comparing multiple variate tokens. With a single variate there is only one token and attention is trivially uninformative. For univariate problems, architectures such as PatchTST or N-BEATS are generally more appropriate.
Does iTransformer require special preprocessing or normalisation?
The original paper evaluates iTransformer with standard normalisation and notes that instance-level normalisation strategies such as RevIN are complementary. Because different variates can have very different scales, per-instance or per-channel normalisation before embedding is strongly recommended in practice and its omission is a common source of degraded results.
Sources
- 1.Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., & Long, M. (2024). iTransformer: Inverted transformers are effective for time series forecasting. ICLR.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 2). iTransformer. ScholarGate. https://scholargate.app/deep-learning/itransformer