한국정보과학회논문지:컴퓨팅의 실제 및 레터 (Journal of KIISE:Computing Practices and Letters)

한국정보과학회 (Korean Institute of Information Scientists and Engineers)
권호 표지

객체지향 메트릭을 이용한 결함 예측 모형의 임계치 설정에 관한 실험

An Experiment for Determining Threshold of Defect Prediction Models using Object Oriented Metrics

  • 김윤규 (부산대학교 컴퓨터공학과) ;
  • 채흥석 (부산대학교 컴퓨터공학과)
  • 발행 : 2009.12.15
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

소프트웨어의 결함을 예측하고 검증과 확인 활동을 통하여 효율적인 자원을 관리하기 위하여 많은 연구에서 결함 예측 모형을 제안하고 있다. 하지만 기존의 연구는 예측율이 최대 효과를 보이는 임계치에 결함 예측 모형의 예측율을 평가하고 있다. 이는 측정 시스템의 결함 정보를 알고 있는 가정하에서 평가가 이루어지는 것이기 때문에 실제 결함 정보를 알 수 없는 시스템에서는 최적의 임계치를 결정할 수 없다. 그러므로 임계치 선정의 중요성을 확인하기 위하여 본 연구에서는 결함 예측 모형으로 타 시스템의 결함을 예측하는 비교 실험을 하였다. 실험은 기존에 제안된 3개의 결함 예측 모형과 4개의 시스템을 대상으로 하였고 결함 예측 모형의 임계치별 예측의 정확성을 비교하였다. 실험결과에서 임계치는 모형의 예측율과 높은 관련이 있었지만 실제 결함 정보가 확인 안 되는 시스템에 대하여 결함을 예측하는 경우에는 임계치를 선정할 수 없음을 확인하였다. 따라서 결함 예측 모형을 타 시스템에 적용하기 위하석 임계치 선정에 관한 추후 연구가 필요함을 확인하였다.

To support an efficient management of software verification and validation activities, many defect prediction models have been proposed based on object oriented metrics. In order to apply defect prediction models, we need to determine a threshold value. Because we cannot know actually where defects are, it is difficult to determine threshold. Therefore, we performed a series of experiments to explore the issue of determining a threshold. In the experiments, we applied defect prediction models to other systems different from the system used in building the prediction model. Specifically, we have applied three models - Olague model, Zhou model, and Gyimothy model - to four different systems. As a result, we found that the prediction capabilities varied considerably with a chosen threshold value. Therefore, we need to perform a study on the determination of an appropriate threshold value to improve the applicably of defect prediction models.

키워드

참고문헌

  1. H.M. Olague, L.H Etzkorn, S. Gholston and S. Quattlebaum, “Empirical validation of three soft-ware metrics suites to predict fault-proneness of object-oriented classes developed using highly ite-rative or agile software development processes,” IEEE Transactions on Software Engineering, vol.33, no.6, pp.402-419, 2007 https://doi.org/10.1109/TSE.2007.1015
  2. T. Gyimothy, R. Ferenc and I. Siket, “Empirical validation of object-oriented metrics on open source software for fault prediction,” IEEE Transactions on Software Engineering, vol.31, no.10, pp.897-910, 2006 https://doi.org/10.1109/TSE.2005.112
  3. Y. Zhou and H. Leung, "Empirical analysis of object-oriented design metrics for predicting high and low severity faults," IEEE Transactions on Software Engineering, vol.32, no.10, pp.771-789, 2006 https://doi.org/10.1109/TSE.2006.102
  4. A. Marcus, D. Poshyvanyk and R. Ferenc, “Using the conceptual cohesion of classes for fault pre-diction in object-oriented systems,” IEEE Trans-actions on Software Engineering, vol.34, no.2, pp. 287-300, 2008 https://doi.org/10.1109/TSE.2007.70768
  5. SW. Menard, Applied logistic regression analysis, Sage Publications Inc, pp.128, 2002
  6. LC. Briand, WL. Melo and J. Wust, "Assessing the applicability of fault-proneness models across object-oriented software projects," IEEE Transactions on Software Engineering, vol.28, no.7, pp.706-720, 2002 https://doi.org/10.1109/TSE.2002.1019484
  7. K.El. Emam, S. Benlarbi, N. Goel and SN. Rai, “The confounding effect of class size on the vali-dity ofobject-oriented metrics,” IEEE Transactions on Software Engineering, vol.27, no.7, pp. 630-650, 2001 https://doi.org/10.1109/32.935855
  8. VR.Basili, LC. Briand and WL.Melo, “A validation of object-oriented design metrics as qualityindi-cators,” IEEE Transactions on Software Engineering, vol.22, no.10, pp.751-761, 1996 https://doi.org/10.1109/32.544352
  9. S. Lessmann, B. Baesens, C. Mues and S. Pietsch, “Benchmarking classification models for software defect prediction: A proposed framework and novel findings,” IEEE Transactions on Software Engineering, vol.34, no.4. pp.485-496, 2008 https://doi.org/10.1109/TSE.2008.35
  10. SR. Chidamber, CF. Kemerer and C. MIT, “A metrics suite for object oriented design,” IEEE Transactions on Software Engineering, vol.20. no.6, pp.476-493, 1994 https://doi.org/10.1109/32.295895
  11. Eclipse., eclipse project archived downloads, http://archive.eclipse.org/eclipse/downloads
  12. Borland., Together 2007, http://www.borland.c-om/us/products/together/index.html
  13. NASA IV&V FACILITY., Metrics Data Program, http://mdp.ivv.nasa.gov/repository.html
  14. PROMISE., PROMISE data sets, http://promise-data.org/
  15. T. Zimmermann, R. Premraj and A. Zeller, “Pre-dicting defects for eclipse,” in Predictor Models in Software Engineering(PROMISE'07): ICSE Workshops 2007, International Workshop on, pp.9-15, 2007 https://doi.org/10.1109/PROMISE.2007.10
  16. S. Watanabe, H. Kaiya and K. Kaijiri, “Adapting a fault prediction model to allow inter langua-gereuse,” in Proceedings of the 4th international workshop on Predictor models in scftware engin-eering, pp.19-24, 2008 https://doi.org/10.1145/1370788.1370794
  17. T. Fawcett, “An introduction to ROC analysis,” Pattern Recognition Letters, vol.27, no.8, pp.861-874, 2006 https://doi.org/10.1016/j.patrec.2005.10.010