Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Improving Deep Learning Models Considering the Time Lags between Explanatory and Response Variables

  • Chaehyeon Kim (Dept. of Computer and Information Science, University of Pennsylvania) ;
  • Ki Yong Lee (Dept. of Computer Science, Sookmyung Women's University)
  • Received : 2022.03.14
  • Accepted : 2022.07.16
  • Published : 2024.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

A regression model represents the relationship between explanatory and response variables. In real life, explanatory variables often affect a response variable with a certain time lag, rather than immediately. For example, the marriage rate affects the birth rate with a time lag of 1 to 2 years. Although deep learning models have been successfully used to model various relationships, most of them do not consider the time lags between explanatory and response variables. Therefore, in this paper, we propose an extension of deep learning models, which automatically finds the time lags between explanatory and response variables. The proposed method finds out which of the past values of the explanatory variables minimize the error of the model, and uses the found values to determine the time lag between each explanatory variable and response variables. After determining the time lags between explanatory and response variables, the proposed method trains the deep learning model again by reflecting these time lags. Through various experiments applying the proposed method to a few deep learning models, we confirm that the proposed method can find a more accurate model whose error is reduced by more than 60% compared to the original model.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2021R1A2C1012543).

References

  1. C. Kim, E. Ryoo, and K. Y. Lee, "Deep learning model for identifying the time lag between explanatory variables and response variable in regression analysis," in Proceedings of Annual Conference of the Korea Information Processing Society, Yeosu, Korea, 2021, pp. 868-871. https://doi.org/10.3745/PKIPS.y2021m11a.868 
  2. J. Wu, "Prediction of birth rate in China under three-child policy based on neural network," in Proceedings of 2022 7th International Conference on Intelligent Computing and Signal Processing (ICSP), Xi'an, China, 2022, pp. 1652-1655. https://doi.org/10.1109/ICSP54964.2022.9778548 
  3. K. Kim and S. Jeon, "Scenario analysis of fertility in Korea using the fertility rate prediction model," The Korean Journal of Applied Statistics, vol. 28, no. 4, pp. 685-701, 2015. https://doi.org/10.5351/KJAS.2015.28.4.685 
  4. T. Hayduk and M. Walker, "The effect of advertising on sales and brand equity in small sport businesses," Sport Marketing Quarterly, vol. 30, no. 3, pp. 178-192, 2021. http://doi.org/10.32731/SMQ.303.0921.02 
  5. Q. Sun, C. Liu, T. Chen, and A. Zhang, "A weighted-time-lag method to detect lag vegetation response to climate variation: a case study in Loess Plateau, China, 1982-2013," Remote Sensing, vol. 13, no. 5, article no. 923, 2021. https://doi.org/10.3390/rs13050923 
  6. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, pp. 1-9. 
  7. H. Gu, Y. Wang, S. Hong, and G. Gui, "Blind channel identification aided generalized automatic modulation recognition based on deep learning," IEEE Access, vol. 7, pp. 110722-110729, 2019. https://doi.org/10.1109/ACCESS.2019.2934354 
  8. M. Aamir, Z. Rahman, W. A. Abro, M. Tahir, and S. M. Ahmed, "An optimized architecture of image classification using convolutional neural network," International Journal of Image, Graphics and Signal Processing, vol. 11, no. 10, pp. 30-39, 2019. https://doi.org/10.5815/ijigsp.2019.10.05 
  9. S. Miao, Z. J. Wang, and R. Liao, "A CNN regression approach for real-time 2D/3D registration," IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1352-1363, 2016. https://doi.org/10.1109/TMI.2016.2521800 
  10. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735 
  11. I. O. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, et al., "MLP-mixer: an all-MLP architecture for vision," Advances in Neural Information Processing Systems, vol. 34, pp. 24261-24272, 2021. 
  12. H. Han, "Residual learning based CNN for gesture recognition in robot interaction," Journal of Information Processing Systems, vol. 17, no. 2, pp. 385-398, 2021. https://doi.org/10.3745/JIPS.01.0072 
  13. J. Kim, J. Park, M. Shin, J. Lee, and N. Moon, "The method for generating recommended candidates through prediction of multi-criteria ratings using CNN-BiLSTM," Journal of Information Processing Systems, vol. 17, no. 4, pp. 707-720, 2021. https://doi.org/10.3745/JIPS.02.0159 
  14. M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio, "Binarized neural networks: training deep neural networks with weights and activations constrained to+ 1 or -1," 2016 [Online]. Available: https://arxiv.org/abs/1602.02830 
  15. J. Han and M. Kamber, Data Mining: Concepts and Techniques. San Francisco, CA: Morgan Kaufmann, 2001. 
  16. S. Gold and A. Rangarajan, "Softmax to softassign: neural network algorithms for combinatorial optimization," Journal of Artificial Neural Networks, vol. 2, no. 4, pp. 381-399, 1996. 
  17. Statistics Korea, "KOSIS (Korean Statistical Information Service)," c2024 [Online]. Available: http://kosis.kr