Browse > Article
http://dx.doi.org/10.3837/tiis.2020.04.007

A Hybrid Cloud Testing System Based on Virtual Machines and Networks  

Chen, Jing (Shandong Provincial Key Laboratory of Computer Networks, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences))
Yan, Honghua (Development department, Shandong GreenPower Technology Co., Ltd)
Wang, Chunxiao (Shandong Provincial Key Laboratory of Computer Networks, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences))
Liu, Xuyan (Network Management Department, China Telecom Corporation Limited, Shandong Branch company)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.14, no.4, 2020 , pp. 1520-1542 More about this Journal
Abstract
Traditional software testing typically uses many physical resources to manually build various test environments, resulting in high resource costs and long test time due to limited resources, especially for small enterprises. Cloud computing can provide sufficient low-cost virtual resources to alleviate these problems through the virtualization of physical resources. However, the provision of various test environments and services for implementing software testing rapidly and conveniently based on cloud computing is challenging. This paper proposes a multilayer cloud testing model based on cloud computing and implements a hybrid cloud testing system based on virtual machines (VMs) and networks. This system realizes the automatic and rapid creation of test environments and the remote use of test tools and test services. We conduct experiments on this system and evaluate its applicability in terms of the VM provision time, VM performance and virtual network performance. The experimental results demonstrate that the performance of the VMs and virtual networks is satisfactory and that this system can improve the test efficiency and reduce test costs through rapid virtual resource provision and convenient test services.
Keywords
cloud testing; cloud computing; TaaS; virtual machine; virtual network;
Citations & Related Records
연도 인용수 순위
  • Reference
1 X. Xu, H. Jin, S. Wu, L. Tang and Y. Wang, "URMG: enhanced CBMG-based method for automatically testing web applications in the cloud," Tsinghua Science and Technology, vol. 19, no. 1, pp. 65-75, February, 2014.   DOI
2 C. Tao and J. Gao, "Cloud-based mobile testing as a service," International Journal of Software Engineering and Knowledge Engineering, vol. 26, no. 1, pp. 147-152, 2016.   DOI
3 D. Tao, Z. Lin and C. Lu, "Cloud platform based automated security testing system for mobile internet," Tsinghua Science and Technology, vol. 20, no. 6, pp. 537-544, December, 2015.   DOI
4 C.-H. Liu, "A cloud platform for compatibility testing of android multimedia applications," in Proc. of International Conference on Frontier Computing, pp. 169-178, July 12-14, 2017.
5 C. Guo, S. Zhu, T. Wang and H. Wang, "FeT: hybrid cloud-based mobile bank application testing," in Proc. of IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C), pp. 21-26, July 16-20, 2018.
6 M. Ficco, R. Pietrantuono and S. Russo, "Hybrid simulation and test of vessel traffic systems on the cloud," IEEE Access, vol. 6, pp. 47273-47287, August, 2018.   DOI
7 H. Kim, A. Ahmad, J. Hwang and et al, "IoT-TaaS:towards a prospective IoT testing framework," IEEE Access, vol. 6, pp. 15480-15493, 2018.   DOI
8 W.-T. Tsai and G. Qi, "Integrated fault detection and test algebra for combinatorial testing in TaaS (Testing-as-a-Service)," Simulation Modeling Practice and Theory, vol. 68, pp. 108-124, November, 2016.   DOI
9 S. Zhang and B. Pi, "Mobile functional test on TaaS environment," in Proc. of 9th IEEE International Symposium on Service-Oriented System Engineering (SOSE), pp. 315-320, March 30-April 3, 2015.
10 C. Tao and J. Gao, "On building a cloud-based mobile testing infrastructure service system," Journal of System and Software, vol. 124, pp. 39-55, February, 2017.   DOI
11 S. Ali and H. Li, "Moving Software Testing to the Cloud: An Adoption Assessment Model Based on Fuzzy Multi-Attribute Decision Making Algorithm," in Proc. of 2019 IEEE 6th IEEE International Conference on Industrial Engineering and Applications (ICIEA), pp. 382-386, April 12-15, 2019.
12 A.-R. Eid, H. Aloran, M. Salah and et. al., "A mutation-based model to rank testing as a service (TaaS) providers in cloud computing," in Proc. of the International Conference on Internet of things and Cloud Computing, pp. 1-5, March 22-23, 2016.
13 R. R. Oliveira, R. M. Oliveira, A. S. Oliveira, "Impact of the Vendor Lock-in Problem on Testing as a Service (TaaS)," in Proc. of 2017 IEEE International Conference on Cloud Engineering (ICCE), pp. 190-196, April 4-8, 2017.
14 Parasoft, "SOAtest," https://www.parasoft.com/products/soatest.
15 Akamai, "CloudTest," https://www.akamai.com/cn/zh/products/performance/cloudtest.jsp.
16 Sauce Labs, "Continuous Testing Cloud," https://saucelabs.com.
17 S. Jain, D. Srivastava, "Testing as a service (TaaS) on cloud: needs and challenges," International Journal of Advanced Research in Computer Science & Technology, vol. 2, no. 2, pp. 335-340, April-June, 2014.
18 Microsoft, "Xamarin Test Cloud," https://testcloud.xamarin.com.
19 Q. Li, D. L. Ke, and X. L. Wang, "Brief survey on cloud testing," Application Research of Computers, vol. 29, no. 12, pp. 4401-4406, December, 2012.   DOI
20 J. Gao, X. Bai, W.-T. Tsai, "Testing as a service (TaaS) on clouds," in Proc. of IEEE 7th International Symposium on Service-Oriented System Engineering (SOSE), pp. 212-223, March 25-28, 2013.
21 A. Bertolino, L. Nautiyal and P. Malik, "Annotated buzzwords and key references for software testing in the cloud," in Proc. of IEEE International Conference on Computing, Communication and Automation, pp. 893-900, May 5-6, 2017.
22 M. Kaur, "Testing in the cloud: new challenges," in Proc. of IEEE International Conference on Computing, Communication and Automation, pp. 742-746, April 29-30, 2016.
23 B. Kavitha and P. Varalakshmi, "Performance analysis of virtual machines and docker containers," in Proc. of International Conference on Data Science Analytics and Applications, pp. 99-113, January 4-6, 2018.
24 Baidu, "Baidu MTC," http://mtc.baidu.com.
25 Testin, "cloud testing service platform," http://www.testin.cn.
26 J. Chen, C. Wang, F. Liu and Y. Wang, "Research and implementation of a software online testing platform model based on cloud computing," in Proc. of 5th International Conference on Advanced Cloud and Big Data, pp. 87-93, August 13-16, 2017.
27 Openstack, "Open source software for creating private and public clouds," https://www.openstack.org.
28 UnixBench, "A benchmark suite for unix-like systems," https://www.ostechnix.com/unixbench-benchmark-suite-unix-like-systems.
29 V. Mittal, L. Nautiyal and M. Mittal, "Cloud testing-the future of contemporary software testing," in Proc. of International Conference on Next Generation Computing and Information Systems (ICNGCIS), pp. 131-136, December 11-12, 2017.
30 P. Harikrishna and A. Amuthan, "A survey of testing as a service in cloud computing," in Proc. of 2016 international conference on computer communication and informatics (ICCCI), pp. 1-5, January 7-9, 2016.
31 A. S. Sathe, R. Kulkarni, "Study of testing as a service (Taas) -cost effective framework for taas in cloud environment," International Journal of Application or Innovation in Engineering & Management, vol. 2, no. 5, pp. 240-243, May 2013.
32 L. Cao, Y. Jiang, C. M. Gan, Y. C. Zhang and G. Q. Chen, "Construction of software testing platform on cloud computing," New Technology of Library and Information Service, vol. 11, pp. 34-39, November, 2012.
33 Alibaba, "MQC," http://mqc.aliyun.com.
34 Tecent, "WeTest," https://wetest.qq.com.
35 A. Bertolino, A. Calabro, D. A. Guglielmo, et al, "When the testing gets tough, the tough get ElasTest," in Proc. of 40th ACM/IEEE International Conference on Software Engineering (ICSE), pp. 17-20, May 27-June 3, 2018.
36 L. Riungu-Kalliosaari, O. Taipale, K. Smolander and I. Richardson, "Adoption and use of cloud-based testing in practice," Software Quality Journal, vol. 24, no. 2, pp. 337-364, October, 2016.   DOI
37 T. Banzai, H. Koizumi, R. Kanbayashi, T. Imada, T. Hanawa and M. Sato, "D-cloud: design of a software testing environment for reliable distributed systems using cloud computing technology," in Proc. of 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing, pp. 631-636, May 17-20, 2010.
38 A. Gambi, W. Hummer and S. Dustdar, "Automated testing of cloud-based elastic systems with AUToCLES," in Proc. of 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 714-717, November 11-15, 2013.
39 W. T. Lo, R. K. Sheu, S. M. Yuan and G.-H. Luo, "Automatic test environment deployment and service testing model in the cloud," Journal of Internet Technology, vol. 17, no. 3, pp. 599-607, May, 2016.
40 C. Chiang, C. Chang, H. Chen, Y. Chen, S. Yuan and C. Wang, "ATP: a browser-based distributed testing service platform," International Computer Symposium, pp. 192-197, December 15-17, 2016.
41 M. Yan, H. Sun, X. Wang and X. Liu, "Building a TaaS platform for web service load testing," in Proc. of IEEE International Conference on Cluster Computing, pp. 576-579, September 24-28, 2012.
42 S. Lee, Y. Lin, K. Lin and J. You, "Composing and delivering heterogeneous web testing software as a composite web testing service," International Computer Symposium (ICS), pp. 605-610, December 15-17, 2016.
43 C. Li and H. Shih, "A cloud testing platform and its methods based on essential cloud characteristics," in Proc. of International Conference on Machine Learning and Cybernetics (ICMLC), pp. 163-169, July 12-15, 2015.
44 X. P. Ding, H. Hou, S. He, Z. G. Chen and D. H. Guo, "Performance study of cloud testing platform based on openstack," Software, vol. 36, no. 1, pp. 6-10, March, 2015.