Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Test Case Grouping and Filtering for Better Performance of Spectrum-based Fault Localization

결함위치식별 기법의 성능 향상을 위한 테스트케이스 그룹화 및 필터링 기법

  • 김정호 (성균관대학교 전자전기컴퓨터공학과) ;
  • 이은석 (성균관대학교 컴퓨터공학과)
  • Received : 2016.03.11
  • Accepted : 2016.05.17
  • Published : 2016.08.15
⟨ Previous Next ⟩

Abstract

Spectrum-based fault localization (SFL) method assigns a suspicious ratio. The statement is strongly affected by a failed test case compared to a passed test case. A failed test case assigns a suspicious ratio while a passed test case reduces some parts of assigned suspicious ratio. In the absence of a failed test case, it is impossible to localize the fault. Thus, a failed test case is very important for fault localization. However, spectrum-based fault localization has difficulty in reflecting the unique characteristics of a failed test because a failed test case and a passed test case are input at the same time to calculate a suspicious ratio. This paper supplements for this limitation and suggests a test case grouping method for more accurate fault localization. In addition, this paper suggested a filtering method considering test efficiency and verified the effectiveness by applying 65 algorithms. In 90 % of whole methods, the accuracy was improved by 13% and the effectiveness was improved by 72% based on EXAM score.

스펙트럼 기반 결함위치식별 기법은 성공 테스트케이스 대비 실패 테스트케이스에 영향을 많이 받은 스테이트먼트에 의심도를 통계적으로 부여하는 기법이다. 이 기법에서 실패 테스트케이스를 지나간 스테이트먼트에는 의심도를 부여하고 성공 테스트케이스를 지나간 스테이트먼트는 부여된 의심도 일부를 줄이는 역할을 한다. 그러므로 실패 테스트케이스의 역할이 매우 중요하며 부재 시 결함의 위치를 찾을 수 없기도 하다. 하지만 이 기법에서 실패, 성공 테스트케이스를 동시에 입력하여 의심도를 계산하기 때문에 실패 테스트케이스의 고유 특성을 반영할 수 없다는 한계점을 가지고 있다. 본 논문에서는 이와 같은 한계점을 보완하여 보다 정확한 결함위치식별을 도와줄 수 있는 테스트케이스 그룹화기법을 제안한다. 또한, 테스트 효율성을 고려한 필터링 기법을 제안하고 이들을 65개의 알고리즘에 적용해 실효성을 검증한다. EXAM score기준으로 전체의 90% 기법에서 정확도 13%, 효율성이 72% 향상되었다.

Keywords

Acknowledgement

Supported by : 한국연구재단, 정보통신기술진흥센터

References

  1. David Landsberg, Hana Chockler, Daniel Kroening, and Matt Lewis, "Evaluation of Measures for Statistical Fault Localisation and an Optimising Scheme," Fundamental Approaches to Software Engineering, Springer Berlin Heidelberg, pp. 115-129, 2015.
  2. Costa, Pedro, Joao Gabriel Silva, and Henrique Madeira, "Practical and representative faultloads for large-scale software systems," Journal of Systems and Software 103, pp. 182-197, 2015. https://doi.org/10.1016/j.jss.2015.02.001
  3. Noor, Tanzeem, and Hadi Hemmati, "Test case analytics: Mining test case traces to improve riskdriven testing," Software Analytics (SWAN) IEEE 1st International Workshop on. IEEE, pp. 13-16, 2015.
  4. Jeongho Kim, Jonghee Park and Eunseok Lee, "A New Spectrum-based Fault Localization with the Technique of Test Case Optimization," Journal of Information Science and Engineering (JISE), Vol. 32, No. 1 Jan, 2016, in press.
  5. Abreu, Rui, Peter Zoeteweij, and Arjan JC Van Gemund, "On the accuracy of spectrum-based fault localization," Testing: Academic and Industrial Conference Practice and Research Techniques-MUTATION (TAICPART-MUTATION), IEEE, pp. 89-98, 2007.
  6. Xie, Xiaoyuan, W. Eric Wong, Tsong Yueh Chen, and Baowen Xu, "Metamorphic slice: An application in spectrum-based fault localization," Information and Software Technology 55.5, pp. 866-879, 2013. https://doi.org/10.1016/j.infsof.2012.08.008
  7. Wong, W. Eric, and Yu Qi, "Effective program debugging based on execution slices and interblock data dependency," Journal of Systems and Software 79.7, pp. 891-903, 2006. https://doi.org/10.1016/j.jss.2005.06.045
  8. Wong, W. Eric, Debroy, V., Ruizhi Gao, and Yihao Li, "The dstar method for effective software fault localization," Reliability, IEEE Transactions on 63.1, pp. 290-308, 2014. https://doi.org/10.1109/TR.2013.2285319
  9. Wong, W. Eric, Vidroha Debroy, and Byoungju Choi, "A family of code coverage-based heuristics for effective fault localization," Journal of Systems and Software 83.2, pp. 188-208, 2010. https://doi.org/10.1016/j.jss.2009.09.037
  10. Naish, Lee, Hua Jie Lee, and Kotagiri Ramamohanarao, "A model for spectra-based software diagnosis," ACM Transactions on software engineering and methodology (TOSEM) 20.3, Article 11, 2011.
  11. Xie, Xiaoyuan, Tsong Yueh Chen, Fei-Ching Kuo, and Baowen Xu, "A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization," ACM Transactions on Software Engineering and Methodology (TOSEM) 22.4, Article 31, 2013.
  12. Kim, Jeongho, Jonghee Park, and Eunseok Lee, "A new hybrid algorithm for software fault localization," Proc. of the 9th International Conference on Ubiquitous Information Management and Communication. ACM, 2015.
  13. Kim, Jeongho, and Eunseok Lee, "Empirical evaluation of existing algorithms of spectrum based fault localization," Information Networking (ICOIN), 2014 International Conference on. IEEE, pp. 346-351, 2014.
  14. Misherghi, Ghassan, and Zhendong Su, "HDD: hierarchical delta debugging," Proc. of the 28th international conference on Software engineering, pp. 142-151, 2006.
  15. Groce, Alex, Mohammad Amin Alipour, Chaoqiang Zhang, Yang Chen, and John Regehr, "Cause reduction: delta debugging, even without bugs," Software Testing, Verification and Reliability 26, No. 1 pp. 40-68, 2016. https://doi.org/10.1002/stvr.1574
  16. Jia, Yue, and Mark Harman, "An analysis and survey of the development of mutation testing," Software Engineering, IEEE Transactions on 37.5, pp. 649-678, 2011. https://doi.org/10.1109/TSE.2010.62
  17. Debroy, Vidroha, and W. Eric Wong, "Combining mutation and fault localization for automated program debugging," Journal of Systems and Software 90, pp. 45-60, 2014. https://doi.org/10.1016/j.jss.2013.10.042
  18. Papadakis, Mike, and Yves Le Traon, "Metallaxis-FL: mutation-based fault localization," Software Testing, Verification and Reliability, Vol. 25, No. 5-7, pp. 605-628, 2013. https://doi.org/10.1002/stvr.1509
  19. Gopinath, Divya, Razieh Nokhbeh Zaeem, and Sarfraz Khurshid, "Improving the effectiveness of spectra-based fault localization using specifications," Automated Software Engineering (ASE), Proceedings of the 27th IEEE/ACM International Conference on. IEEE, pp. 40-49, 2012.
  20. Bekkouche, Mohammed, Helene Collavizza, and Michel Rueher, "LocFaults: A new flowdriven and constraint-based error localization approach," Symposium on Applied Computing (SAC), ACM., Vol. 15, 2015.
  21. Khoshnood, Sepideh, Markus Kusano, and Chao Wang, "Concbugassist: Constraint solving for diagnosis and repair of concurrency bugs," International Symposium on Software Testing and Analysis, pp. 165-176, 2015.
  22. Ke, Yalin, Kathryn T. Stolee, Claire Le Goues and Yuriy Brun, "Repairing programs with semantic code search," Proc. of the 30th IEEE/ACM International Conference On Automated Software Engineering (ASE), pp. 295-306, 2015.
  23. Yoo, Shin, "Evolving human competitive spectrabased fault localisation techniques," Search Based Software Engineering, Springer Berlin Heidelberg, pp. 244-258, 2012.
  24. Yoo, Shin, and Mark Harman, "Regression testing minimization, selection and prioritization: a survey," Software Testing, Verification and Reliability 22.2, pp. 67-120, 2012. https://doi.org/10.1002/stv.430
  25. Daniel, Patrick, "On improving the accuracy of spectrum-based fault localization," Doctoral dissertation, Swinburne University of Technology, 2014.
  26. Do, Hyunsook, Sebastian Elbaum, and Gregg Rothermel, "Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact," Empirical Software Engineering, 10.4, pp. 405-435 2005. https://doi.org/10.1007/s10664-005-3861-2
  27. Bayraktar, "ELA: An automated statistical fault localization technique," Doctoral dissertation, Middle East Technical University, 2015.
  28. Software-artifact Infrastructure Repository, [Online]. Available: http://sir.unl.edu/portal/index.php