Journal of the Korea Society for Simulation (한국시뮬레이션학회논문지)

The Korea Society for Simulation (한국시뮬레이션학회)
권호 표지
DOI QR코드

DOI QR Code

Time Series Data Analysis using WaveNet and Walk Forward Validation

WaveNet과 Work Forward Validation을 활용한 시계열 데이터 분석

  • Received : 2021.10.08
  • Accepted : 2021.11.25
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Deep learning is one of the most widely accepted methods for the forecasting of time series data which have the complexity and non-linear behavior. In this paper, we investigate the modification of a state-of-art WaveNet deep learning architecture and walk forward validation (WFV) in order to forecast electric power consumption data 24-hour-ahead. WaveNet originally designed for raw audio uses 1D dilated causal convolution for long-term information. First of all, we propose a modified version of WaveNet which activates real numbers instead of coded integers. Second, this paper provides with the training process with tuning of major hyper-parameters (i.e., input length, batch size, number of WaveNet blocks, dilation rates, and learning rate scheduler). Finally, performance evaluation results show that the prediction methodology based on WFV performs better than on the traditional holdout validation.

복잡하고 비선형적인 특징을 갖는 시계열 데이터를 예측하기 위해 딥러닝 기법이 널리 사용되고 있다. 본 연구에서는 최근에 개발된 WaveNet을 개선하고 워크포워드 검증 기법을 적용하여 전력 소비량 데이터를 24시간 이전에 예측하고자 한다. 원래 WaveNet은 오디오 데이터 예측에 사용하고자 고안되었으며, 장기간의 데이터를 효과적으로 예측하기 위해 1차원 팽창인과 합성곱(1D dilated causal convolution)을 사용한다. 먼저, WaveNet이 부호화된 정수 값이 아니라 실수 값을 출력하여 전력 데이터를 예측하기 적합하도록 개선하였다. 다음으로 학습 과정에 적용된 하이퍼파라미터(입력 기간, 배치 크기, WaveNet 블록 개수, 팽창 비율, 학습률 변경)를 조정하여 적절한 성능을 나타내도록 하였다. 마지막으로 성능 평가를 통해 전통적인 홀드아웃 검증 기법보다 본 연구에서 사용한 워크포워드 검증 기법이 전력 소비량 데이터 예측에 우수함 성능을 나타냄을 확인하였다.

Keywords

References

  1. Bergstra, J., Bardenet, R., Bengio, Y., and Kegl, B., "Algorithms for Hyper-parameter Otimization", Proc. of 25th Annual Conference on Neural Information Processing Systems, pp. 1-9, 2011
  2. Cho K, Merrienboer B. van, Gulcehre C, Bahdanau, D., Bougares, F,. Schwenk, H., and Bengio, Y. "Learning Prase Rpresentations sing RNN Encoder-Decoder for Statistical Machine Translation", Proc. of Empirical Methods in Natural Language Processing. pp. 1724-1734. 2014.
  3. Clements M. P., Franses P. H., and Swanson N. R., "Forecasting Economic and Financial Time-Series with Non-linear Models". International Journal of Forecast, vol. 20, no. 2, pp. 169-183, 2004. https://doi.org/10.1016/j.ijforecast.2003.10.004
  4. Hochreiter, S. and Schmidhuber, J., "Long Short-Term Memory", Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997 https://doi.org/10.1162/neco.1997.9.8.1735
  5. Hong, T., Gui, M., Baran, M., and Willis, H. L., "Modeling and Forecasting Hourly Electric Load by Multiple Linear Regression with Interactions", Proc. of the IEEE Power and Energy Society General Meeting, pp. 1-8, 2010.
  6. Hong, T., Wang, P., and Willis, H. L., "A Naive Multiple Linear Regression Benchmark for Short Term Load Forecasting", Proc. of the IEEE Power and Energy Society General Meeting, vol. 2, pp. 1-6, 2011.
  7. Karim, A. and Ariffin, S., "Electricity Load Forecasting in UTP Using Moving Averages and Exponential Smoothing Techniques", Applied Mathematical Sciences, vol. 7, no. 77-80, pp. 4003-4014, 2013. https://doi.org/10.12988/ams.2013.33149
  8. Livieris, I. E., Emmanuel, P., and Panagiotis, P., "A CNN-LSTM Model for Gold Price TimeSeries Forecasting", Neural Computing and Applications. vol. 32, no. 23, pp. 17351-17360, 2020. https://doi.org/10.1007/s00521-020-04867-x
  9. Masters, D. and Luschi, C., "Revisiting Small Batch Training for Deep Neural Networks", 1-18. http://arxiv.org/abs/1804.07612, 2018
  10. Ngoc T. T., van Dai, L., and Phuc D. T., "Grid Search of Multilayer Perceptron Based on the Walk-Forward Validation methodology", International Journal of Electrical and Computer Engineering, vol. 11, no, 2, pp. 1742-1751, 2021.
  11. Oord, A. van den, Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K., "WaveNet: A Generative Model for Raw Audio", http://arxiv.org/abs/1609.03499, 2016.
  12. Tealab, A., "Time Series Forecasting Using Artificial Neural Networks Methodologies: A Systematic Review." Future Computing and Informatics Journal, vol. 3, no. 2, pp. 334-340. 2018. https://doi.org/10.1016/j.fcij.2018.10.003