论文标题
K变量时间序列是值得k词：长期多元时间序列的香草变压器体系结构的演变预测
A K-variate Time Series Is Worth K Words: Evolution of the Vanilla Transformer Architecture for Long-term Multivariate Time Series Forecasting
论文作者
论文摘要
多元时间序列预测（MTSF）是许多现实世界应用中的基本问题。最近，变压器已成为MTSF的事实上解决方案，尤其是对于长期案例。但是，除了一个远期操作外，几乎没有仔细验证现有的MTSF变压器体系结构中的基本配置。在这项研究中，我们指出，MTSF变压器体系结构中的当前令牌化策略忽略了变压器的令牌统一感应偏置。因此，香草MTSF变形金刚努力捕获时间序列中的细节并表现出劣等性能。基于此观察，我们对香草MTSF变压器的基本结构进行了一系列演变。我们与解码器结构和嵌入一起改变了有缺陷的令牌化策略。令人惊讶的是，进化的简单变压器架构非常有效，它成功地避免了香草MTSF变压器中过度平滑的现象，实现了更详细和准确的预测，甚至大大超过了最先进的变压器，这些变压器已很好地设计了MTSF。
Multivariate time series forecasting (MTSF) is a fundamental problem in numerous real-world applications. Recently, Transformer has become the de facto solution for MTSF, especially for the long-term cases. However, except for the one forward operation, the basic configurations in existing MTSF Transformer architectures were barely carefully verified. In this study, we point out that the current tokenization strategy in MTSF Transformer architectures ignores the token uniformity inductive bias of Transformers. Therefore, the vanilla MTSF transformer struggles to capture details in time series and presents inferior performance. Based on this observation, we make a series of evolution on the basic architecture of the vanilla MTSF transformer. We vary the flawed tokenization strategy, along with the decoder structure and embeddings. Surprisingly, the evolved simple transformer architecture is highly effective, which successfully avoids the over-smoothing phenomena in the vanilla MTSF transformer, achieves a more detailed and accurate prediction, and even substantially outperforms the state-of-the-art Transformers that are well-designed for MTSF.
