iTransformer: Inverted Transformer for Multivariate Time-Series Forecasting
iTransformer (Inverted Transformer for Forecasting) · Also known as: Inverted Transformer, iTransformer for Time Series, Inverted Attention Transformer, Ters Transformer
iTransformer is a deep-learning architecture for multivariate time-series forecasting introduced by Liu et al. at ICLR 2024. Its defining idea is to invert the conventional Transformer tokenisation strategy: instead of treating each time step as a token, iTransformer treats each variate (sensor channel or feature series) as a single token whose embedding encodes the full observed look-back window. Self-attention is then applied across variates to capture inter-series dependencies, while a feed-forward network within each token learns temporal patterns.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
iTransformer is well suited to multivariate forecasting tasks where cross-variable dependencies are informative and the number of variates is large relative to the horizon. It assumes the look-back window is long enough to provide meaningful per-variate embeddings and that inter-series correlations are stable across time. It is less appropriate for univariate forecasting (the inversion provides no advantage) or when variate counts are very small. Alternatives include PatchTST for patch-based temporal tokenisation, Crossformer for fine-grained cross-time-cross-variate attention, or N-BEATS and TimesNet for different inductive biases.
Strengths & limitations
- Directly models inter-variate correlations through the attention mechanism without hand-crafted channel mixing
- Computationally efficient for high-variate datasets because attention sequence length scales with N (variates), not T (time steps)
- The feed-forward network retains the ability to capture variate-level temporal patterns despite the inverted token structure
- Simple, clean architectural modification to standard Transformer that is easy to implement and reproduce
- Offers no direct advantage in purely univariate settings where cross-variate attention is vacuous
- Per-variate token embeddings conflate the entire look-back window into a single vector, potentially losing fine-grained temporal positional information
- Performance may degrade if inter-variate relationships are non-stationary or change over time
- Like most Transformer-based forecasters, requires substantial training data and GPU resources to outperform simpler linear baselines reliably
Frequently asked
What makes iTransformer different from a standard Transformer applied to time series?
A standard Transformer tokenises each time step, so attention captures temporal position dependencies. iTransformer tokenises each variate (channel), so attention captures inter-series dependencies instead. Temporal patterns are handled by the feed-forward sub-layer applied within each variate token. This architectural inversion is the sole but consequential change.
Is iTransformer suitable for univariate forecasting?
Not in its standard form. The inverted attention mechanism gains its benefit from comparing multiple variate tokens. With a single variate there is only one token and attention is trivially uninformative. For univariate problems, architectures such as PatchTST or N-BEATS are generally more appropriate.
Does iTransformer require special preprocessing or normalisation?
The original paper evaluates iTransformer with standard normalisation and notes that instance-level normalisation strategies such as RevIN are complementary. Because different variates can have very different scales, per-instance or per-channel normalisation before embedding is strongly recommended in practice and its omission is a common source of degraded results.
Sources
- Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., & Long, M. (2024). iTransformer: Inverted transformers are effective for time series forecasting. ICLR. link ↗
How to cite this page
ScholarGate. (2026, June 2). iTransformer (Inverted Transformer for Forecasting). ScholarGate. https://scholargate.app/en/deep-learning/itransformer
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- CrossformerDeep learning↔ compare
- PatchTSTDeep learning↔ compare