Advancing Time Series Forecasting: Innovative Machine Learning Approaches
Loading...
Authors
Zhang, Yifan
Issue Date
2024
Type
Dissertation
Language
en_US
Keywords
Deep Learning , Recurrent Neural Networks , Time Series , Transformer
Alternative Title
Abstract
The importance of multivariate time series (MTS) data is increasingly acknowledged in scientific and technological domains. Effectively modeling historical time series records and accurately forecasting future events remain critical challenges within the field of artificial intelligence. Two key issues in this domain require further investigation. First, the probability distribution of time series data may vary over time. Second, effectively and efficiently modeling temporal dependencies remains a significant challenge. This dissertation tackles the problem of MTS forecasting using advanced deep neural network techniques and is organized into three parts. The first part provides an overview of the historical development of time series forecasting and highlights recent advancements. We also discuss potential future directions for addressing this challenge and outline our motivations. In the second part, we explore the use of recurrent neural networks (RNN) to model the evolving distributions of time series data under extreme conditions. We introduce a framework that integrates RNN with other machine learning models to forecast normal and extreme time series instances independently. Additionally, we present a novel extreme event adaptive gated recurrent unit (eGRU) architecture that models both normal and extreme events using the same RNN cell. The effectiveness of the first approach is demonstrated using real-world datasets obtained from our collaborators, while the second approach is validated using publicly available benchmarks. In the third part, we investigate the application of transformer neural networks for modeling temporal patterns in time series data. We propose a pyramid architecture comprising multiple transformer encoder-decoder pairs to capture temporal dependencies at various scales. Additionally, we investigate the efficiency of the self-attention mechanism, the core component of the transformer, whose quadratic computational complexity limits its use in MTS forecasting by constraining the size of the look-back window. To address this limitation, we introduce a sequence-adaptive sparse attention mechanism that reduces computational complexity to a linear scale.
Description
Citation
Publisher
License
CC BY-NC