Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Improving Performance of ART with Iterative Partitioning using Test Case Distribution Management

테스트 케이스 분포 조절을 통한 IP-ART 기법의 성능 향상 정책

  • 신승훈 (아주대학교 정보통신공학과) ;
  • 박승규 (아주대학교 정보 및 컴퓨터공학부) ;
  • 최경희 (아주대학교 정보 및 컴퓨터공학부)
  • Published : 2009.06.15
Download PDF
⟨ Previous Next ⟩

Abstract

The Adaptive Random Testing(ART) aims to improve the performance of traditional Random Testing(RT) by reducing the number of test cases to find the failure region which is located in the input domain. Such enhancement can be obtained by efficient selection algorithms of test cases. The ART through Iterative Partitioning(IP-ART) is one of ART techniques and it uses an iterative input domain partitioning method to improve the performance of early-versions of ART which have significant drawbacks in computation time. And the IP-ART with Enlarged Input Domain(EIP-ART), an improved version of IP-ART, is known to make additional performance improvement with scalability by expanding to virtual test space beyond real input domain of IP-ART. The EIP-ART algorithm, however, have the drawback of heavy cost of computation time to generate test cases mainly due to the virtual input domain enlargement. For this reason, two algorithms are proposed in this paper to mitigate the computation overhead of the EIP-ART. In the experiments by simulations, the tiling technique of input domain, one of two proposed algorithms, showed significant improvements in terms of computation time and testing performance.

적응적 랜덤 테스팅(Adaptive Random Testing, ART)은 테스트 케이스의 효율적인 선택을 통해 순수 랜덤 테스팅(Random Testing, RT)보다 더 적은 수의 테스트 케이스를 이용해 입력 도메인 내의 오류를 찾는 것을 목적으로 한다. ART 기법 중 하나인 입력 도메인 반복 분할 기법(ART through Iterative Partitioning, IP-ART)은 초기 ART 기법의 단점인 많은 연산량을 입력 도메인 분할에 의해 효율적으로 개선되도록 하였으며, 입력 도메인 확장을 이용한 IP-ART(IP-ART with Enlarged Input Domain, EIP-ART)는 IP-ART의 테스트 케이스 분포 특징을 이용하여 추가적인 성능 향상과 확장성을 제공하였다. 하지만 EIP-ART는 입력 도메인 확장에 따라 발생하는 부하로 인해 테스트 케이스 생성에 오랜 시간을 요구하기 때문에 이의 개선이 필요하다. 따라서 본 논문에서는 두 가지의 추가 부하를 유발하지 않는 테스트 케이스 분포 조절 기법을 제안하고, 이들의 성능 개선 가능성을 실험을 통해 확인하였으며, 실험 결과, 제안된 두 기법 중 입력 도메인 타일링 기법이 모든 환경에서 더 우수한 성능 및 확장성을 갖는 것으로 확인되었다.

Keywords

References

  1. R. Hamlet, 'Random testing' Encyclopedia of Software Engineering, pp. 970-978, Wiley, 1994
  2. G. J. Myers, 'The Art of Software Testing,' 2nd ED., John Wiley & Sons, 2004
  3. H. Bati, L. Giakoumakis, S. Herbert, A. Surna, "A genetic approach for random testing of database systems," in Proceedings of the 33rd International Conference on Very Large Data Bases (VLDB '07), pp. 1243-1251, Sep. 2007
  4. T. Yoshikawa, K. Shimura, T. Ozawa, 'Random program generator for Java JIT compiler test system,' in Proceedings of the 3rd International Conference on Quality Software (QSIC '03), pp. 20-23, Nov. 2003
  5. J. Regehr, 'Random Testing of Interrupt-Driven Software,' in Proceedings of the 5th ACM international conference on Embedded software (EMSOFT '05), pp. 290-298, Sep. 2005. https://doi.org/10.1145/1086228.1086282
  6. J. Duran and S. Ntafos, 'An Evaluation of Random Testing,' IEEE Transactions on Software Engineering, volume 10, no. 4, pp. 438–444, Jul. 1984 https://doi.org/10.1109/TSE.1984.5010257
  7. D. Hamlet, 'When Only Random Testing will do,' in Proceedings of the First International Workshop on Random. Testing (RT '06), pp. 1-9, Jul. 2006 https://doi.org/10.1145/1145735.1145737
  8. D. Hamlet and R. Taylor, 'Partition Testing Does Not Inspire Confidence,' IEEE Transactions on Software Engineering, Vol. 16, issue 12, pp. 1402-1411, Dec. 1990 https://doi.org/10.1109/32.62448
  9. K. P. Chan, T. Y. Chen, and D. Towey, 'Restricted Random Testing,' in Proceedings of the 7th European Conference on Software Quality (ECSQ 2002), vol. 2349 of Lecture Notes in Computer Science, pp. 321-330, Springer, Jun. 2002
  10. T. Y. Chen, D. H. Huang, Z. Q. Zhou, 'Adaptive Random Testing Through Iterative Partitioning,' in Proceedings of the 11th International Conference on Reliable Software Technologies (Ada-Europe 2006), volume 4006 of Lecture Notes in Computer Science, pp. 155-166. Springer-Verlag, Jun. 2006 https://doi.org/10.1145/1188895.1188903
  11. T. Y. Chen, H. Leung, and I. K. Mak, 'Adaptive Random Testing,' in Proceedings of the 9th Asian Computing Science Conference (ASIAN 2004), vol. 3321 of Lecture Notes in Computer Science, pp. 320-329. Springer, Dec. 2004
  12. J. Mayer, 'Adaptive Random Testing with Randomly Translated Failure Region,' in Proceedings of the 1st International Workshop on Random Testing (RT 2006), pp. 70-77, Jul. 2006 https://doi.org/10.1145/1145735.1145746
  13. J. Mayer, C. Schneckenburger, 'Adaptive Random Testing with Enlarged Input Domain,' in Proceedings of the Sixth International Conference on Quality Software (QSIC '06), pp. 251-258, Aug. 2006 https://doi.org/10.1109/QSIC.2006.8
  14. 신승훈, 박승규, '입력 도메인 확장을 이용한 반복 분할 기반의 적응적 랜덤 테스팅 기법', 정보처리학회논문지D, volume 15D, no. 4, pp. 531-540, 2008년 8월 https://doi.org/10.3745/KIPSTD.2008.15-D.4.531
  15. 이광규, 신승훈, 박승규, '반복 분할 기반의 적응적 랜덤 테스팅 향상 기법', 대한전자공학회 논문지, 제45권, CI편, 제5호, pp. 180-191, 2008년 9월
  16. F. T. Chan, T. Y. Chen, I. K. Mak, and Y. T. Yu, 'Proportional Sampling Strategy: Guidelines for Software Testing Practitioners,' Information and Software Technology, vol. 38, no. 12, pp. 775- 782, Dec. 1996 https://doi.org/10.1016/0950-5849(96)01103-2
  17. T. Chen, T. Tse, Y. Yu, 'Proportional sampling strategy: a compendium and some insights,' The Journal of Systems and Software, vol. 58, pp. 65-81, Aug. 2001 https://doi.org/10.1016/S0164-1212(01)00028-0