Non-stationary Transformer

Non-stationary Transformer is a Transformer-based time-series forecasting architecture introduced by Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long at NeurIPS 2022. It addresses a fundamental tension in applying Transformers to real-world time series: over-stationarization during preprocessing strips out non-stationary signals that carry predictive information, while raw non-stationary inputs cause attention to collapse. The model resolves this through series stationarization paired with a novel de-stationary attention mechanism that restores the original temporal distribution in predictions.