Browse > Article
http://dx.doi.org/10.3837/tiis.2022.06.006

Cloud Task Scheduling Based on Proximal Policy Optimization Algorithm for Lowering Energy Consumption of Data Center  

Yang, Yongquan (Department of Computer Science and technology (Ocean University of China))
He, Cuihua (Department of Computer Science and technology (Ocean University of China))
Yin, Bo (Department of Computer Science and technology (Ocean University of China))
Wei, Zhiqiang (Department of Computer Science and technology (Ocean University of China))
Hong, Bowei (Department of Computer Science and technology (Ocean University of China))
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.6, 2022 , pp. 1877-1891 More about this Journal
Abstract
As a part of cloud computing technology, algorithms for cloud task scheduling place an important influence on the area of cloud computing in data centers. In our earlier work, we proposed DeepEnergyJS, which was designed based on the original version of the policy gradient and reinforcement learning algorithm. We verified its effectiveness through simulation experiments. In this study, we used the Proximal Policy Optimization (PPO) algorithm to update DeepEnergyJS to DeepEnergyJSV2.0. First, we verify the convergence of the PPO algorithm on the dataset of Alibaba Cluster Data V2018. Then we contrast it with reinforcement learning algorithm in terms of convergence rate, converged value, and stability. The results indicate that PPO performed better in training and test data sets compared with reinforcement learning algorithm, as well as other general heuristic algorithms, such as First Fit, Random, and Tetris. DeepEnergyJSV2.0 achieves better energy efficiency than DeepEnergyJS by about 7.814%.
Keywords
cloud computing; cloud task scheduling; deep reinforcement learning; energy consumption; proximal policy optimization;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 J. Yu, B. Zhang, Z. Kuang, D. Lin, and J. Fan, "iPrivacy: Image Privacy Protection by Identifying Sensitive Objects via Deep Multi-Task Learning," IEEE Transactions on Information Forensics and Security, vol. 12, no. 5, pp. 1005-1016, May 2017.   DOI
2 V. Mnih et al., "Playing Atari with Deep Reinforcement Learning," arXiv:1312.5602 [cs], Dec. 2013.
3 A. F. S. Devaraj, M. Elhoseny, S. Dhanasekaran, E. L. Lydia, and K. Shankar, "Hybridization of firefly and Improved Multi-Objective Particle Swarm Optimization algorithm for energy efficient load balancing in Cloud Computing environments," Journal of Parallel and Distributed Computing, vol. 142, pp. 36-45, Aug. 2020.   DOI
4 D. Ding, X. Fan, Y. Zhao, K. Kang, Q. Yin, and J. Zeng, "Q-learning based dynamic task scheduling for energy-efficient cloud computing," Future Generation Computer Systems, vol. 108, pp. 361-371, Jul. 2020.   DOI
5 W. McKinney, "Data Structures for Statistical Computing in Python," in Proc. of the 9th Python in Science Conference, pp. 56-61, 2010.
6 Martin Abadi et al., "TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems," 2015. [Online]. Available: https://www.tensorflow.org/
7 Sukhpal Singh Gill et al., "AI for Next Generation Computing: Emerging Trends and Future Directions," Internet of Things, 2022.
8 Y. Yin, Y. Xu, W. Xu, M. Gao, L. Yu, and Y. Pei, "Collaborative Service Selection via Ensemble Learning in Mixed Mobile Network Environments," Entropy, vol. 19, no. 7, Jul. 2017.
9 J. D. Hunter, "Matplotlib: A 2D Graphics Environment," Computing in Science & Engineering, vol. 9, no. 03, pp. 90-95, May 2007.   DOI
10 V. Mnih et al., "Human-level control through deep reinforcement learning," Nature, vol. 518, no. 7540, pp. 529-533, Feb. 2015.   DOI
11 J. Stuart, Norvig, and Peter, Artificial Intelligence: A Modern Approach, 1995.
12 L. Y. Zuo and Z. B. Cao, "Review of scheduling research in cloud computing," Application Research of Computers, vol. 29, no. 11, pp. 4023-4027, 2012.   DOI
13 C. J. C. H. Watkins, "Learning from delayed rewards," Ph.D. dissertation, King's College, Cambridge United Kingdom, 1989.
14 Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May 2015.   DOI
15 J. Yu, Z. Kuang, B. Zhang, W. Zhang, D. Lin, and J. Fan, "Leveraging Content Sensitiveness and User Trustworthiness to Recommend Fine-Grained Privacy Settings for Social Image Sharing," IEEE Transactions on Information Forensics and Security, vol. 13, no. 5, pp. 1317-1332, May 2018.   DOI
16 S. Seth and N. Singh, "Dynamic heterogeneous shortest job first (DHSJF): a task scheduling approach for heterogeneous cloud computing systems," Int. j. inf. tecnol., vol. 11, no. 4, pp. 653-657, Dec. 2019.   DOI
17 R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning," Mach Learn, vol. 8, no. 3, pp. 229-256, May 1992.   DOI
18 "Alibaba Cluster Trace Program," Alibaba, 2021. Accessed: Jan. 04, 2022. [Online]. Available: https://github.com/alibaba/clusterdata/blob/4221e02342dd01fd30a9800b19b7f365a3fd5ac8/cluster-trace-v2018/trace_2018.md
19 H. Peng, W.-S. Wen, M.-L. Tseng, and L.-L. Li, "Joint optimization method for task scheduling time and energy consumption in mobile cloud computing environment," Applied Soft Computing, vol. 80, pp. 534-545, Jul. 2019.   DOI
20 J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, "Trust Region Policy Optimization," in Proc. of the 32nd International Conference on Machine Learning, pp. 1889-1897, Jun. 2015.
21 "SimPy," Team SimPy, 2020. [Online]. Available: https://simpy.readthedocs.io/en/latest/index.html
22 C. He, Y. Yang, and B. Hong, "Cloud Task Scheduling Based on Policy Gradient Algorithm in Heterogeneous Cloud Data Center for Energy Consumption Optimization," in Proc. of 2020 International Conference on Internet of Things and Intelligent Applications (ITIA), pp. 1-5, Nov. 2020.
23 J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal Policy Optimization Algorithms," arXiv:1707.06347 [cs], Aug. 2017.
24 C. Lu, K. Ye, G. Xu, C.-Z. Xu, and T. Bai, "Imbalance in the cloud: An analysis on Alibaba cluster trace," in Proc. of 2017 IEEE International Conference on Big Data (Big Data), pp. 2884-2892, Dec. 2017.
25 S. van der Walt, S. C. Colbert, and G. Varoquaux, "The NumPy Array: A Structure for Efficient Numerical Computation," Computing in Science Engineering, vol. 13, no. 2, pp. 22-30, Mar. 2011.