KNOM Review

The Korean Institute of Commucations and Information Sciences (한국통신학회)
권호 표지
DOI QR코드

DOI QR Code

A study on Deep Q-Networks based Auto-scaling in NFV Environment

NFV 환경에서의 Deep Q-Networks 기반 오토 스케일링 기술 연구

  • Lee, Do-Young (Pohang University of Science and Technology, Dept. of Computer Science and Engineering) ;
  • Yoo, Jae-Hyoung (Pohang University of Science and Technology, Dept. of Computer Science and Engineering) ;
  • Hong, James Won-Ki (Pohang University of Science and Technology, Dept. of Computer Science and Engineering)
  • Received : 2020.11.01
  • Accepted : 2020.12.22
  • Published : 2020.12.31
⟨ Previous Next ⟩

Abstract

Network Function Virtualization (NFV) is a key technology of 5G networks that has the advantage of enabling building and operating networks flexibly. However, NFV can complicate network management because it creates numerous virtual resources that should be managed. In NFV environments, service function chaining (SFC) composed of virtual network functions (VNFs) is widely used to apply a series of network functions to traffic. Therefore, it is required to dynamically allocate the right amount of computing resources or instances to SFC for meeting service requirements. In this paper, we propose Deep Q-Networks (DQN)-based auto-scaling to operate the appropriate number of VNF instances in SFC. The proposed approach not only resizes the number of VNF instances in SFC composed of multi-tier architecture but also selects a tier to be scaled in response to dynamic traffic forwarding through SFC.

5G 네트워크의 핵심 기술 중 하나인 네트워크 기능 가상화 (NFV, Network Function Virtualization)는 유연하고 민첩한 네트워크 구축 및 운용을 가능하게 만드는 장점이 있다. 하지만, 한편으로는 수 많은 가상 자원을 생성하기 때문에 네트워크 관리를 복잡하게 만드는 원인이 된다. 일반적으로, NFV 환경에서는 가상 네트워크 기능(VNF, Virtual Network Function)들로 구성된 서비스 펑션 체이닝 (SFC, Service Function Chaining)을 통해 일련의 네트워크 기능들을 트래픽에 적용한다. 따라서 서비스 요구사항을 만족시킬 수 있도록 동적으로 SFC에 알맞은 양의 컴퓨팅 자원 또는 인스턴스를 할당하는 것이 필요하다. 본 논문에서는 SFC에서 적절한 수의 VNF 인스턴스를 운용하기 위해 강화학습 알고리즘의 하나인 Deep Q-Networks (DQN)을 이용한 Auto-scaling 방법을 제안한다. 제안하는 방법은 SFC로 유입되는 트래픽의 증감에 따라 SFC를 구성하는 다계층 (Multi-tier) 구조에서 스케일링(Scaling)이 필요한 계층을 선택하고, 스케일링을 통해 효과적으로 VNF 인스턴스들 개수를 조절한다.

Keywords

Acknowledgement

본 연구는 과학기술정보통신부 및 정보통신기술진흥센터의 정보통신·방송 연구개발사업의 일환으로 수행하였음.(No. 2018-0-00749, 인공지능 기반 가상 네트워크 관리기술 개발)

References

  1. R. Boutaba, M. A. Salahuddin, N. Limam, S. Ayoubi, N. Shahriar, F. Estrada-Solano, and O. M. Caicedo, "A comprehensive survey on machine learning for networking: evolution, applications and research opportunities," Journal of Internet Services and Applications, vol. 9, no. 1, p. 16, 2018. https://doi.org/10.1186/s13174-018-0087-2
  2. 김기현, 최미정, "SDN, NFV, Edge-Computing을 이용한 데이터 중심 네트워크 기술 동향 분석", KNOM Review, Vol. 22, No. 3, Dec. 2019, pp. 1-12.
  3. 김희곤, 이도영, 유재형, 홍원기, "기계학습 기반의 가상 네트워크 기능 자원 수요 예측 방법", KNOM Review, Vol. 21, No. 2, Dec. 2018, pp. 1-9. https://doi.org/10.22670/KNOM.2018.21.2.1
  4. 박수현, 김희곤, 홍지범, 유재형, 홍원기, "기계학습 기반 VNF 최적 배치 예측 기술연구", KNOM Review, Vol. 23, No. 1, Aug. 2020, pp. 34-42. https://doi.org/10.22670/KNOM.2020.23.1.34
  5. S. Lange, H.-G. Kim, S.-Y. Jeong, H. Choi, J.-H. Yoo, and J. W.-K. Hong, "Machine learning-based prediction of vnf deployment decisions in dynamic networks," in 2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS). IEEE, 2019, pp. 1-6.
  6. S. Lange, H.-G. Kim, S.-Y. Jeong, H.-Y. Choi, J.-H. Yoo, and J. W.- K. Hong, "Predicting vnf deployment decisions under dynamically changing network conditions," in 2019 15th International Conference on Network and Service Management (CNSM). IEEE, 2019, pp. 1-9.
  7. PARK, Suhyun, et al. Machine Learning-based Optimal VNF Deployment. In: 2020 21st Asia-Pacific Network Operations and Management Symposium (APNOMS). IEEE, 2020. p. 67-72.
  8. M. Ghaznavi, A. Khan, N. Shahriar, K. Alsubhi, R. Ahmed, and R. Boutaba, "Elastic virtual network function placement," in 2015 IEEE 4th International Conference on Cloud Networking (CloudNet). IEEE, 2015, pp. 255-260.
  9. X. Wang, C. Wu, F. Le, A. Liu, Z. Li, and F. Lau, "Online vnf scaling in datacenters," in 2016 IEEE 9th International Conference on Cloud Computing (CLOUD). IEEE, 2016, pp. 140-147.
  10. A. Xie, H. Huang, X. Wang, Z. Qian, and S. Lu, "Online vnf chain deployment on resource-limited edges by exploiting peer edge devices," Computer Networks, vol. 170, p. 107069, 2020. https://doi.org/10.1016/j.comnet.2019.107069
  11. M. Wajahat, A. Gandhi, A. Karve, and A. Kochut, "Using machine learning for black-box autoscaling," in 2016 Seventh International Green and Sustainable Computing Conference (IGSC). IEEE, 2016, pp. 1-8.
  12. N. Roy, A. Dubey, and A. Gokhale, "Efficient autoscaling in the cloud using predictive models for workload forecasting," in 2011 IEEE 4th International Conference on Cloud Computing. IEEE, 2011, pp. 500- 507.
  13. H. Ghanbari, B. Simmons, M. Litoiu, C. Barna, and G. Iszlai, "Optimal autoscaling in a iaas cloud," in Proceedings of the 9th international conference on Autonomic computing, 2012, pp. 173-178.
  14. A. Evangelidis, D. Parker, and R. Bahsoon, "Performance modelling and verification of cloud-based auto-scaling policies," Future Generation Computer Systems, vol. 87, pp. 629-638, 2018. https://doi.org/10.1016/j.future.2017.12.047
  15. W. Iqbal, A. Erradi, M. Abdullah, and A. Mahmood, "Predictive autoscaling of multi-tier applications using performance varying cloud resources," IEEE Transactions on Cloud Computing, 2019.
  16. L. Yazdanov and C. Fetzer, "Lightweight automatic resource scaling for multi-tier web applications," in 2014 IEEE 7th International Conference on Cloud Computing. IEEE, 2014, pp. 466-473.
  17. Z. Wang, C. Gwon, T. Oates, and A. Iezzi, "Automated cloud provisioning on aws using deep reinforcement learning," arXiv preprint arXiv:1709.04305, 2017.
  18. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., "Human-level control through deep reinforcement learning," nature, vol. 518, no. 7540, pp. 529-533, 2015. https://doi.org/10.1038/nature14236
  19. DPNM, Network Intelligent Project. [Online]. Available:https://github.com/dpnm-ni/ni-auto-scaling-module-public
  20. DPNM, Network Intelligent Project. [Online]. Available:https://github.com/dpnm-ni/ni-mano
  21. C. Amza, E. Cecchet, A. Chanda, A. L. Cox, S. Elnikety, R. Gil, J. Marguerite, K. Rajamani, and W. Zwaenepoel, "Specification and implementation of dynamic web site benchmarks," in 5th Workshop on Workload Characterization, no. CONF, 2002.
  22. D. Krishnamurthy and J. Rolia, "Predicting the qos of an electronic commerce server: Those mean percentiles," ACM SIGMETRICS Performance Evaluation Review, vol. 26, no. 3, pp. 16-22, 1998. https://doi.org/10.1145/306225.306232
  23. M. Andreolini, E. Casalicchio, M. Colajanni, and M. Mambelli, "A cluster-based web system providing differentiated and guaranteed services," Cluster Computing, vol. 7, no. 1, pp. 7-19, 2004. https://doi.org/10.1023/B:CLUS.0000003940.34740.be