Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

A Test Case Prioritization Technique via Value-Based Approach

가치기반 접근법을 통한 테스트 케이스 우선순위 기법

  • 박현철 (삼성탈레스 기술연구소) ;
  • 류호연 ((주)포스데이타 SW테스트팀) ;
  • 백종문 (한국과학기술원 정보과학기술대학)
  • Published : 2009.05.15
Download PDF
⟨ Previous Next ⟩

Abstract

Software, once developed, has a long life and evolves through numerous additions and modifications because of the faults, the changes in user requirements, the changes in environments, and so forth. With the evolution of the software, assuring the quality of the software is getting more difficult because of numerous versions of the software. Meanwhile, regression testing has been used to support the software testing activities and assure acquiring appropriate quality through several versions of software. Regression testing, however, is too expensive because it requires lots of test cases executions and the number of test cases increases sharply as the software evolves. For this reason, several techniques are suggested to help conducting regression testing then test case prioritization technique is understood the most effective and efficient technique to support regression testing. In this paper, we propose an approach, Historical Value-Based Approach, which is based on the use of historical information to estimate the current cost and fault severity for cost-cognizant test case prioritization. As a result of the proposed approach, software testers who perform regression testing prioritize their test cases more effectively so that the test effectiveness of them can be improved in terms of APFDc.

한번 개발된 소프트웨어는 긴 수명을 가지며, 결함이나 사용자 요구사항의 변경, 환경의 변화등의 원인으로 인한 수많은 모듈의 수정을 통해 진화한다. 소프트웨어의 진화와 더불어, 이러한 많은 모듈의 추가와 수정에 의한 소프트웨어의 다양한 버전의 생성은 소프트웨어 품질 향상을 가로막는 주요 요인중의 하나이다. 한편 회귀테스트는 이러한 소프트웨어의 변경에 따른 초점을 둔 테스트이지만, 소프트웨어가 여러 버전을 거쳐 진화함에 따라 많은 수의 테스트 케이스의 추가 및 수정과 더불어 테스트 케이스의 전체 숫자가 급격히 증가함에 따라 비용이 많이 들기에 쉽게 수행하지 못하는 테스트이다. 이에 테스트케이스 우선순위화 기법이 등장하여 회귀테스트의 수행을 지원하였다. 그러나, 테스트 케이스 우선순위화기법에서 사용되는 우선순위의 기준이 가치중립적이거나 가치를 고려하더라도 특정한 단일 요인만을 고려하여 그 활용이 제한적이었다. 논문에서, 우리는 비용과 결함 심각도에 기반한 히스토리컬 가치 기반의 접근법을 제시하며, 이는 기존의 비용 인식 테스트 케이스 우선순위화 기법에서의 현재의 비용 및 결함 심각도를 예측하기 위하여 히스토리컬 정보를 사용하는 접근법이다. 본 논문의 공헌으로서, 제안된 접근법은 테스트의 이해관계자들이 어떻게 히스토리컬 가치가 가치의 관점에서 테스트 효과성의 향상을 위해 사용되고 있는가를 알 수 있도록 돕는 다는데 있다. 결과적으로, 회귀 테스트를 수행하는 소프트웨어 테스터들은 그들의 테스트 케이스를 보다 더 효과적으로 우선순위 화할 수 있기에 그들의 테스트를 통한 테스트 효과성은 향상될 수 있다.

Keywords

References

  1. Gregg Rothermel, R. Untch, C. Chu, and Mary J. Harrold, 'Prioritizing Test Cases for Regression Testing,' IEEE Transactions on Software Engineering, Vol.27, No.10, pp. 929-948, October 2001 https://doi.org/10.1109/32.962562
  2. Gregg Rothermel, Sebastian Elbaum, A.G. Malishevsky, P. Kallakuri, and X. Qiu, 'On Test Suite Composition and Cost-Effective Regression Testing,' ACM Transactions on Software Engineering and Methodology, Vol.13, No.3, pp. 227-331, July 2004
  3. Jung-Min Kim and Adam Porter, 'A History-Based Test Prioritization Technique for Regression Testing in Resource Constrained Environments,' Proceedings of the International Conference on Software Engineering (ICSE`02), May 2002
  4. Alexey G. Malishevsky, Joseph R. Ruthruff, Gregg Rothermel, and Sebastian Elbaum, 'Cost-cognizant Test Case Prioritization,' Technical Report TRUNL-CSE-2006-0004, University of Nebraska-Lincoln, March 2006
  5. Institute of Electrical and Electronics Engineers (IEEE). IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. New York, NY: 1990
  6. Jung-Min Kim, Adam Porter, and Gregg Rothermel, 'An Empirical Study of Regression Test Application Frequency,' Proceedings of the International Conference on Software Engineering (ICSE`00), June 2000
  7. A. Srivastava and J. Thiagarajan, 'Effectively Prioritizing Tests in Development Environment,' Proceedings of the International Symposium on Software Testing and Analysis, July 2002
  8. W.E. Wong, J.R. Horgan, S. London, and H. Agrawal, 'A Study of Effective Regression Testing in Practice,' Proceedings of the International Symposium on Software Reliability Engineering, November 1997
  9. Gregg Rothermel and M.J. Harrold, 'Analyzing Regression Test Selection Techniques,' IEEE Transactions on Software Engineering, Vol.22, No.8, pp. 529-551, August 1996 https://doi.org/10.1109/32.536955
  10. Sebastian Elbaum, Alexey G. Malishevsky, and Gregg Rothermel, 'Test Case Prioritization: A Family of Empirical Studies,' IEEE Transactions on Software Engineering, Vol.28, No.2, pp. 159-182, Feburary 2002 https://doi.org/10.1109/32.988497
  11. Sebastian Elbaum, Alexey Malishevsky, and Gregg Rothermel, 'Prioritizing Test Cases for Regression Testing,' Proceedings of the International Symposium on Software Testing and Analysis, August 2000
  12. Hema Srikanth and Laurie Williams, 'On the Economics of Requirements-Based Test Case Prioritization,' Proceedings of the International Workshop on Economics-Driven Software Engineering Research (EDSER'05), July 2005
  13. Hyunsook Do and Gregg Rothermel, 'A Controlled Experiment Assessing Test Case Prioritization Techniques via Mutation Faults,' Proceedings of the International Conference on Software Maintenance (ICSM`05), 2005
  14. Hyunsook Do and Gregg Rothermel, 'On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques,' IEEE Transactions on Software Engineering, Vol.32, No.9, September 2006
  15. Hyunsook Do, Sebastian Elbaum, and Gregg Rothermel, 'Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and Its Potential Impact,' Empirical Software Engineering, Vol.10, No.4, pp. 405-435, 2005 https://doi.org/10.1007/s10664-005-3861-2
  16. Software-artifact Infrastructure Repository, http://sir.unl.edu
  17. Hyunsook Do, Gregg Rothermel, and A. Kinneer, 'Prioritizing JUnit Test Cases: An Empirical Assessment and Cost-Benefits Analysis,' Empirical Software Engineering, Vol.11, No.1, pp. 33-70, Mar. 2006 https://doi.org/10.1007/s10664-006-5965-8
  18. Hyunsook Do, Gregg Rothermel, and Alex Kinneer, 'Empirical Studies of Test Case Prioritization in a JUnit Testing Environment,' Proceedings of the International Symposium on Software Reliability Engineering (ISSRE'04), November 2004