Crossformer: Cross-Dimension Dependency Transformer for Multivariate Time Series Forecasting
Crossformer (Cross-Dimension Dependency Transformer) · Also known as: Cross-Dimension Dependency Transformer, Crossformer TSF, Çapraz-Boyut Bağımlılık Transformatörü
Crossformer is a Transformer-based architecture for multivariate time series forecasting, introduced by Yunhao Zhang and Junchi Yan at ICLR 2023. Unlike earlier Transformer variants that treat each variate independently, Crossformer explicitly models cross-dimension dependencies alongside temporal patterns. It achieves this through a two-stage attention design — cross-time and cross-dimension — applied over segment-level embeddings organized in a hierarchical encoder, enabling the model to capture both intra-variate dynamics and inter-variate correlations simultaneously.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Crossformer is well-suited for multivariate time series forecasting tasks where inter-variable correlations are expected to carry predictive signal, such as traffic, energy, weather, or financial datasets with many co-evolving channels. It assumes that cross-dimension dependencies are informative and that segment-level temporal abstractions are appropriate. It may underperform simpler linear baselines on datasets where inter-variate structure is weak or where channel independence is a better inductive bias. PatchTST or iTransformer may be preferable alternatives depending on dataset characteristics.
Strengths & limitations
- Jointly models both temporal and cross-dimension dependencies within a unified Transformer framework
- Hierarchical segment merging enables efficient multi-scale temporal representation without extreme sequence length
- Segment-level embedding reduces the quadratic attention cost compared to token-per-timestep approaches
- Explicitly addresses the limitation of channel-independent forecasting models on correlated multivariate data
- On datasets with weak inter-variate correlations, the cross-dimension attention may introduce noise rather than signal
- Segment length is a sensitive hyperparameter that requires tuning per dataset
- Hierarchical merging introduces a fixed coarsening schedule that may not suit all temporal resolutions
- Computational cost scales with both the number of variates and the number of segments, which can be substantial for high-dimensional datasets
Frequently asked
How does Crossformer differ from PatchTST?
PatchTST applies patch-based attention independently per variate (channel independence), making it strong when inter-variable correlations are weak. Crossformer explicitly adds a cross-dimension attention stage to capture inter-variate dependencies, making it more expressive on correlated multivariate datasets but also more prone to overfitting spurious cross-variate patterns.
What is the role of the segment length hyperparameter?
The segment length L_seg controls the granularity of temporal tokens fed into the two-stage attention. Shorter segments capture finer-grained temporal patterns but increase sequence length and computational cost, while longer segments provide coarser summaries with lower cost. It must be tuned per dataset and interacts with the hierarchical merging schedule across encoder layers.
Is Crossformer always better than linear forecasting baselines?
Not necessarily. Several studies have shown that simple linear models such as DLinear can match or exceed complex Transformer-based models on standard benchmarks when inter-variate dependencies are weak or when the forecasting task is largely trend-driven. Crossformer's advantage is more evident on datasets with genuine and predictive cross-dimension structure.
Sources
- Zhang, Y., & Yan, J. (2023). Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. ICLR. link ↗
How to cite this page
ScholarGate. (2026, June 2). Crossformer (Cross-Dimension Dependency Transformer). ScholarGate. https://scholargate.app/en/deep-learning/crossformer
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
Compare side by side →