Browse > Article
http://dx.doi.org/10.5302/J.ICROS.2012.18.11.997

Big Data Analysis of Software Performance Trend using SPC with Flexible Moving Window and Fuzzy Theory  

Lee, Dong-Hun (SAP Labs Korea TIP)
Park, Jong-Jin (Chungwoon University)
Publication Information
Journal of Institute of Control, Robotics and Systems / v.18, no.11, 2012 , pp. 997-1004 More about this Journal
Abstract
In enterprise software projects, performance issues have become more critical during recent decades. While developing software products, many performance tests are executed in the earlier development phase against the newly added code pieces to detect possible performance regressions. In our previous research, we introduced the framework to enable automated performance anomaly detection and reduce the analysis overhead for identifying the root causes, and showed Statistical Process Control (SPC) can be successfully applied to anomaly detection. In this paper, we explain the special performance trend in which the existing anomaly detection system can hardly detect the noticeable performance change especially when a performance regression is introduced and recovered again a while later. Within the fixed number of sampling period, the fluctuation gets aggravated and the lower and upper control limit get relaxed so that sometimes the existing system hardly detect the noticeable performance change. To resolve the issue, we apply dynamically tuned sampling window size based on the performance trend, and Fuzzy theory to find an appropriate size of the moving window.
Keywords
performance anomaly; SPC (Statistical Process Control); big data; fuzzy theory; performance monitoring;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 D. H. Lee, S. K. Cha, and A. H. Lee, "A performance anomaly detection and analysis framework for DBMS development," IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 8, pp. 1345-1360,Aug. 2012.   DOI
2 D. H. Lee, "Performance anomaly detection and management using statistical process control during software development" Journal of KIISE : Software and Applications (in Korean), vol. 39, no. 8, pp. 639-645, Aug. 2012.   과학기술학회마을
3 D. C. Montgomery, Introduction to Statistical Quality Control, 5th Edition. John Wiley & Sons, Inc., 2005.
4 T. Terano, K. Asai, and M. Sugeno, Applied Fuzzy Systems, AP Professional, 1994.
5 J. J. Park and G. S. Choi, "Fuzzy control system," Kyowoosa, 2001.
6 S. Barber, Beyond Performance Testing, http://www-128.ibm. com/developerworks/rational/library/4169.html
7 S. Barber, http://www.logigear.com/newsletter/explanation_ of_performance_testing_on_an_agile_team-part-1.asp
8 M. Woodside, G. Franks, and D. C. Petriu, "The future of software performance engineering," Proc. of International Conference on Software Engineering, 2007 Future of Software Engineering, pp. 171-187, 2007.
9 A. de Vries and B. J. Conlin, "Article: Design and performance of statistical process control charts applied to estrous detection efficiency," Journal of Dairy Science, vol. 86, pp. 1970-1984, 2003.   DOI
10 V. S. Puranik, "CUSUM quality control chart for monitoring energy use performance," Proc. IEEE International Conference on Industrial Engineering and Engineering Management, pp. 1231-1235, Dec. 2007.
11 M. Komuro, "Experiences of applying SPC techniques to software development processes," ICSE '06: Proc. of the 28th international conference on Software engineering, pp. 577-584, 2006.
12 J. W. Cangussu, R. A. DeCarlo, and A. P. Mathur, "Monitoring the software test process using statistical process control: a logarithmic approach," ACM SIGSOFT Software Engineering Notes, vol. 28, no. 5, pp. 158-167, 2003.   DOI