KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Temporal Fusion Transformers and Deep Learning Methods for Multi-Horizon Time Series Forecasting

Temporal Fusion Transformers와 심층 학습 방법을 사용한 다층 수평 시계열 데이터 분석

  • Received : 2021.07.09
  • Accepted : 2021.08.26
  • Published : 2022.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Given that time series are used in various fields, such as finance, IoT, and manufacturing, data analytical methods for accurate time-series forecasting can serve to increase operational efficiency. Among time-series analysis methods, multi-horizon forecasting provides a better understanding of data because it can extract meaningful statistics and other characteristics of the entire time-series. Furthermore, time-series data with exogenous information can be accurately predicted by using multi-horizon forecasting methods. However, traditional deep learning-based models for time-series do not account for the heterogeneity of inputs. We proposed an improved time-series predicting method, called the temporal fusion transformer method, which combines multi-horizon forecasting with interpretable insights into temporal dynamics. Various real-world data such as stock prices, fine dust concentrates and electricity consumption were considered in experiments. Experimental results showed that our temporal fusion transformer method has better time-series forecasting performance than existing models.

시계열 데이터는 주식, IoT, 공장 자동화와 같은 다양한 실생활에서 수집되고 활용되고 있으며, 정확한 시계열 예측은 해당 분야에서 운영 효율성을 높일 수 있어서 전통적으로 중요한 연구 주제이다. 전반적인 시계열 데이터의 향상된 특징을 추출할 수 있는 대표적인 시계열 데이터 분석 방법인 다층 수평 예측은 최근 부가적 정보를 포함하는 시계열 데이터에 내재한 이질성(heterogeneity)까지 포괄적으로 분석에 활용하여 향상된 시계열 예측한다. 하지만 대부분의 심층 학습 기반 시계열 분석 모델들은 시계열 데이터의 이질성을 반영하지 못했다. 따라서 우리는 잘 알려진 temporal fusion transformers 방법을 사용하여 실생활과 밀접한 실제 데이터를 이질성을 고려한 다층 수평 예측에 적용하였다. 결과적으로 주식, 미세먼지, 전기 소비량과 같은 실생활 시계열 데이터에 적용한 방법이 기존 예측 모델보다 향상된 정확도를 가짐을 확인할 수 있었다.

Keywords

Acknowledgement

이 논문은 2020년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임(2020-0-01826, AI 기반 선도적 실전문제해결 연구인재 양성).

References

  1. B. Lim, S. O. Arik, N. Loeff, and T. Pfister, "Temporal fusion transformers for interpretable multi-horizon time series forecasting," arXiv preprint arXiv:1912.09363, 2019.
  2. F. Luna-Perejon, M. J. Dominguez-Morales, and A. Civit-Balcells, "Wearable fall detector using recurrent neural networks," Sensors, Vol.19, No.22, pp.4885, 2019. https://doi.org/10.3390/s19224885
  3. D. Kraft, K. Srinivasan, and G. Biebe, "Deep learning based fall detection algorithms for embeded systems, smartwatches, and IoT devices using accelerometers," Technologies, Vol.8, No.4, pp.72, 2020. https://doi.org/10.3390/technologies8040072
  4. H. Hewamalage, C. Bergmeir, and K. Bandara, "Recurrent neural networks for time series forecasting: Current status and future directions," International Journal of Forecasting, Vol.37, No.1, pp.388-427, 2021. https://doi.org/10.1016/j.ijforecast.2020.06.008
  5. D. Salinas, V Flunkert, and J. Gasthaus, "DeepAR: Probabilistic forecasting with autoregressive recurrent networks," International Journal of Forecasting, Vol.36, No.3, pp.1181-1191, 2020. https://doi.org/10.1016/j.ijforecast.2019.07.001
  6. R. Wen, K. Torkkola, B. Narayanaswamy, and D. Madeka, "A multi-horizon quantile recurrent forecaster," arXiv preprint arXiv:1711.11053, 2017.
  7. Y. Liu, C. Gong, L. Yang, and Y. Chen, "DSTP-RNN: A dualstage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction," Expert Systems with Applications, Vol.143, pp.113082, 2020. https://doi.org/10.1016/j.eswa.2019.113082
  8. C. Fan, Y. Zhang, Y. Pan, X. Li, C. Zhang, and R. Yuan, "Multi-horizon time series forecasting with temporal attention learning," Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.2527-2535, 2019.
  9. C. Favorita, Corporacion favorita grocery sales forecasting competition, 2009.
  10. Yahoo Finance, Carriage Services, Inc., 2021.
  11. X. Liang, T. Zou, B. Guo, and S. Li, "Assessing Beijing's PM2.5 pollution: Severity, weather impact," APEC and Winter heating, Proceedings of Royal Society, Vol.471, No.2182, pp.20150257, 2015.
  12. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, and H. Xiong, "Informer: Beyond efficient transformer for long sequence time-series forecasting," Proceedings of AAAI, 2021.