The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Odds curve for two classification distributions

두 분류 분포를 위한 오즈 곡선

  • Hong, Chong Sun (Department of Statistics, Sungkyunkwan University) ;
  • Oh, Se Hyeon (Department of Statistics, Sungkyunkwan University) ;
  • Oh, Tae Gyu (Department of Statistics, Sungkyunkwan University)
  • 홍종선 (성균관대학교 통계학과) ;
  • 오세현 (성균관대학교 통계학과) ;
  • 오태규 (성균관대학교 통계학과)
  • Received : 2021.02.25
  • Accepted : 2021.03.25
  • Published : 2021.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

The ROC, TOC, and TROC curves, which are visually descriptive methods of exploring the performance of the binary classification model, are implemented with TP, TN, FP, FN which consist of the confusion matrix, as well as their ratios TPR, TNR, FPR, FNR. In this study, we consider two types odds and then propose an odds curve representing these odds. And show the relationship between the odds curve and ROC curve. Based on the odds curve, we propose not only two statistics that measure the discriminant power of the odds curve but also the criteria for validation ratings of the odds curve. According to the shape of the odds curves, two classification distributions can be estimated and a criterion for validation ratings can be determined. The odds curve can be meaningfully used like other visual methods, and two kinds of measures for the discriminant power can be also applied together as an alternative criterion.

이진분류모형의 성능을 탐색하는 시각적인 대표적인 방법인 ROC 곡선과 TOC 곡선 그리고 TROC 곡선은 혼동행렬을 구성하는 TP, TN, FP, FN 그리고 이들의 비율인 TPR, TNR, FPR, FNR으로 구현된다. 본 연구에서는 두 종류의 비율비인 오즈를 고려하여 단위면적인 정사각형에서의 구현하는 오즈 곡선을 제안하고, ROC 곡선과의 관계를 보인다. 오즈 곡선에서 판별력을 측정하는 두 종류의 측도를 제안하고, 오즈 곡선들의 형태를 바탕으로 두 종류의 측도를 이용하여 두 분류 분포의 판단 기준을 설정한다. 본 연구에서 제안한 오즈 곡선은 다른 시각적인 방법 등과 같이 유용하게 사용할 수 있으며, 오즈 곡선의 판별력을 측정하는 두 종류의 측도들은 분류 성능을 판단하는 대안적인 방법으로 같이 이용할 수 있다.

Keywords

References

  1. Altman, D. G. and Bland, J. M. (1994). Diagnostic tests. 1: Sensitivity and specificity, BMJ: British Medical Journal, 308, 1552. https://doi.org/10.1136/bmj.308.6943.1552
  2. Bamber, D. (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph, Journal of Mathematical Psychology, 12, 387-415. https://doi.org/10.1016/0022-2496(75)90001-2
  3. Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition, 30, 1145-1159. https://doi.org/10.1016/S0031-3203(96)00142-2
  4. Centor, R. M. (1991). Signal detectability: the use of ROC curves and their analyses, Medical Decision Making, 11, 102-106. https://doi.org/10.1177/0272989X9101100205
  5. Cho, M. H. and Hong, C. S. (2015). Two optimal threshold criteria for ROC analysis, Journal of the Korean Data and Information Science Society, 26, 255-260. https://doi.org/10.7465/jkdi.2015.26.1.255
  6. Egan, J. P. and Egan, J. P. (1975). Signal Detection Theory and ROC-Analysis, Academic Press.
  7. Engelmann, B., Hayden, E., and Tasche, D. (2003). Testing rating accuracy, Risk, 16, 82-86.
  8. Fawcett, T. (2004). ROC graphs: Notes and practical considerations for researchers, Machine Learning, 31, 1-38.
  9. Fawcett, T. (2006). An introduction to ROC analysis, Pattern Recognition Letters, 27, 861-874. https://doi.org/10.1016/j.patrec.2005.10.010
  10. Green, D. M. and Swets, J. A. (1966). Signal Detection Theory and Psychophysics (Vol. 1), Wiley, New York.
  11. Hanley, J. A. and McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143, 29-36. https://doi.org/10.1148/radiology.143.1.7063747
  12. Hong, C. S. (2021). Confusion plot for the confusion matrix, The Korean Data & Information Science Society, 32, 427-437. https://doi.org/10.7465/jkdi.2021.32.2.427
  13. Hong, C. S. and Jang, D. H. (2020). Partial AUC using the sensitivity and specificity lines, The Korean Journal of Applied Statistics, 33, 541-553. https://doi.org/10.5351/KJAS.2020.33.5.541
  14. Hong, C. S., Joo, J. S., and Choi, J. S. (2010). Optimal thresholds from mixture distributions, The Korean Journal of Applied Statistics, 23, 13-28. https://doi.org/10.5351/KJAS.2010.23.1.013
  15. Hong, C. S., Jung, E. S., and Jung, D. G. (2013). Standard criterion of VUS for ROC surface, The Korean Journal of Applied Statistics, 26, 977-985. https://doi.org/10.5351/KJAS.2013.26.6.977
  16. Hong, C. S., Jung, M. S., and Shin, H. S. (2019a). Partial VUS and optimal thresholds, The Korean Data & Information Science Society, 30, 895-905. https://doi.org/10.7465/jkdi.2019.30.4.895
  17. Hong, C. S., Jung, M. S., and Shin, H. S. (2019b). Three-way partial VUS, The Korean Data & Information Science Society, 30, 445-454. https://doi.org/10.7465/jkdi.2019.30.2.445
  18. Hong, C. S. and Lee, S. J. (2018). TROC curve and accuracy measures, The Korean Data & Information Science Society, 29, 861-872. https://doi.org/10.7465/jkdi.2018.29.4.861
  19. Hong, C. S. and Na, H. R. (2020). Maximum absolute determinant of the confusion matrix, The Korean Data & Information Science Society, 31, 937-950. https://doi.org/10.7465/jkdi.2020.31.5.937
  20. Hong, C. S. and Wu, Z. Q. (2014). Alternative accuracy for multiple ROC analysis, Journal of the Korean Data & Information Science Society, 25, 1521-1530. https://doi.org/10.7465/jkdi.2014.25.6.1521
  21. Hsieh, F. and Turnbull, B. W. (1996). Nonparametric and semiparametric estimation of the receiver operating characteristic curve, Annals of Statistics, 24, 25-40.
  22. Jeni, L. A., Cohn, J. F., and De La Torre, F. (2013). Facing imbalanced data-recommendations for the use of performance metrics. In 2013 Humaine association conference on affective computing and intelligent interaction (pp. 245-251). IEEE.
  23. Joseph, M. P. (2005). A PD validation framework for Basel II internal ratings-based systems, Credit Scoring and Credit Control IV.
  24. Metz, C. E. (1978). Basic principles of ROC analysis. In Seminars in nuclear medicine (Vol. 8, No. 4, pp. 283-298). WB Saunders.
  25. Metz, C. E. and Kronman, H. B. (1980). Statistical significance tests for binormal ROC curves, Journal of Mathematical Psychology, 22, 218-243. https://doi.org/10.1016/0022-2496(80)90020-6
  26. Pepe, M. S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press, Oxford.
  27. Pepe, M. S., Longton, G., Anderson, G. L., and Schummer, M. (2003). Selecting differentially expressed genes from microarray experiments, Biometrics, 59, 133-142. https://doi.org/10.1111/1541-0420.00016
  28. Pontius Jr, R. G. and Si, K. (2014). The total operating characteristic to measure diagnostic ability for multiple thresholds, International Journal of Geographical Information Science, 28, 570-583. https://doi.org/10.1080/13658816.2013.862623
  29. Provost, F. and Fawcett, T. (1997). Analysis and visualization of classifier performance with nonuniform class and cost distributions. In Proceedings of AAAI-97 Workshop on AI Approaches to Fraud Detection & Risk Management, 57-63.
  30. Provost, F. and Fawcett, T. (2001). Robust classification for imprecise environments, Machine Learning, 42, 203-231. https://doi.org/10.1023/A:1007601015854
  31. Sonego, P., Kocsor, A., and Pongor, S. (2008). ROC analysis: applications to the classification of biological sequences and 3D structures, Briefings in Bioinformatics, 9, 198-209. https://doi.org/10.1093/bib/bbm064
  32. Stein, R. M. (2005). The relationship between default prediction and lending profits: Integrating ROC analysis and loan pricing, Journal of Banking & Finance, 29, 1213-1236. https://doi.org/10.1016/j.jbankfin.2004.04.008
  33. Swets, J. A. (1988). Measuring the accuracy of diagnostic systems, Science, 240, 1285-1293. https://doi.org/10.1126/science.3287615
  34. Tasche, D. (2008). Validation of internal rating systems and PD estimates. In The Analytics of Risk Model Validation, Academic Press, 169-196.
  35. Vuk, M. and Curk, T. (2006). ROC curve, lift chart and calibration plot, Metodoloski Zvezki, 3, 89.
  36. Zweig, M. H. and Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine, Clinical Chemistry, 39, 561-577. https://doi.org/10.1093/clinchem/39.4.561