The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Odds curve and optimal threshold

오즈 곡선과 최적분류점

  • Hong, Chong Sun (Department of Statistics, Sungkyunkwan University) ;
  • Oh, Tae Gyu (Department of Statistics, Sungkyunkwan University) ;
  • Oh, Se Hyeon (Department of Statistics, Sungkyunkwan University)
  • 홍종선 (성균관대학교 통계학과) ;
  • 오태규 (성균관대학교 통계학과) ;
  • 오세현 (성균관대학교 통계학과)
  • Received : 2021.05.11
  • Accepted : 2021.06.20
  • Published : 2021.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Various accuracy measures that can be explained on the odds curve are discussed, and an alternative accuracy measure, the maximum square, is proposed based on the characteristics of the odds curve. Thresholds corresponding to these accuracy measures are obtained by considering various probability distribution functions and an illustrative example. Their characteristics are discussed while comparing many kinds of statistics measuring thresholds. Therefore, we can conclude that optimal thresholds could be explored from the odds curve, similar to the ROC curve, and that the maximum square measure can be used as a good accuracy measure that can improve the performance of the binary classification model.

오즈 곡선으로 설명이 가능한 정확도 측도들을 살펴보고, 오즈 곡선의 성질을 바탕으로 대안적인 최대 사각형 정확도 측도를 제안한다. 다양한 확률분포함수와 실증예제를 고려하여 정확도 측도들에 대응하는 분류점을 구하고, 분류점을 측정하는 통계량들을 비교하면서 특징을 토론한다. 그러므로 ROC 곡선 등과 유사하게 오즈 곡선으로부터도 최적분류점들을 발견하고 설명할 수 있으며, 최대사각형 측도는 이진 분류모형의 성능을 향상시킬 수 있는 정확도 측도로 활용할 수 있다.

Keywords

References

  1. Altman DG and Bland JM (1994). Diagnostic tests. 1: Sensitivity and specificity, BMJ: British Medical Journal, 308, 1552. https://doi.org/10.1136/bmj.308.6943.1552
  2. Bamber D (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph, Journal of Mathematical Psychology, 12, 387-415. https://doi.org/10.1016/0022-2496(75)90001-2
  3. Brasil P (2010). Diagnostic Test Accuracy Evaluation for Medical Professionals, Package DiagnosisMed in R.
  4. Bradley AP (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognitions, 30, 1145-1159. https://doi.org/10.1016/S0031-3203(96)00142-2
  5. Centor RN (1991). Signal detectability: The use of ROC curves and their analyses, Medical Decision Making, 11, 102-106. https://doi.org/10.1177/0272989x9101100205
  6. Cho MH and Hong CS (2015). Two optimal threshold criteria for ROC analysis, Journal of the Korean Data & Information Science Society, 26, 255-260. https://doi.org/10.7465/JKDI.2015.26.1.255
  7. Connell FA and Koepsell TD (1985). Measures of gain in certainty from a diagnostic test, American Journal of Epidemiology, 121, 744-753. https://doi.org/10.1093/aje/121.5.744
  8. Egan JP and Egan JP (1975). Signal Detection Theory and ROC-Analysis, Academic press, New York.
  9. Green DM and Swets JA (1966). Signal detection theory and psychophysics (vol. 1), New York.
  10. Hanley JA and McNeil BJ (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143, 29-36. https://doi.org/10.1148/radiology.143.1.7063747
  11. Hong CS (2009). Optimal threshold from ROC and CAP curves, Communications in Statistics-Simulation and Computation, 38, 2060-2072. https://doi.org/10.1080/03610910903243703
  12. Hong CS and Choi JS (2009). Optimal threshold from ROC and CAP curves, The Korean Journal of Applied Statistics, 22, 911-921. https://doi.org/10.5351/KJAS.2009.22.5.911
  13. Hong CS and Jang DH (2020). Validation ratings for the length of the ROC curve, Journal of the Korean Data & Information Science Society, 31, 851-863. https://doi.org/10.7465/jkdi.2020.31.5.851
  14. Hong CS, Joo JS, and Choi JS (2010). Optimal thresholds from mixture distributions, The Korean Journal of Applied Statistics, 23, 13-28. https://doi.org/10.5351/KJAS.2010.23.1.013
  15. Hong CS and Lee SJ (2018). TROC curve and accuracy measures, Journal of the Korean Data & Information Science Society, 29, 861-872. https://doi.org/10.7465/jkdi.2018.29.4.861
  16. Hong CS, Lin MH, Hong SW, and Kim GC (2011). Classification accuracy measures with minimum error rate for normal mixture, Journal of the Korean Data & Information Science Society, 22, 619-630.
  17. Hong CS, Oh SH, and Oh TG (2021). Odds curve for two classification distributions, The Korean Journal of Applied Statistics, 34, 225-238. https://doi.org/10.5351/KJAS.2021.34.2.225
  18. Joseph MP (2005). A PD validation framework for Basel II internal ratings-based systems, Credit Scoring and Credit Control IV
  19. Krzanowski WJ and Hand DJ (2009). ROC Curves for Continuous Data. Chapman & Hall/CRC, Boca Raton.
  20. Lambert J and Lipkovich I (2008). A macro for getting more out of your ROC curve, SAS Global Forum, 231.
  21. Metz CE (1978). Basic principles of ROC analysis, Seminars in Nuclear Medicine, 8, 283-298.
  22. Moses LE, Shapiro D, and Littenberg B (1993). Combining independent studies of a diagnostic test into a summary ROC curve: Data-analytic approaches and some additional considerations, Statistics in Medicine, 12, 1293-1316. https://doi.org/10.1002/sim.4780121403
  23. Pepe MS (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press, Oxford.
  24. Perkins NJ and Schisterman EF (2006). The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve, American Journal of Epidemiology, 163, 670-675. https://doi.org/10.1093/aje/kwj063
  25. Pettitt AN (1976). A two-sample Anderson-Darling rank statistic, Biometrika, 63, 161-168. https://doi.org/10.1093/biomet/63.1.161
  26. Pontius Jr, RG and Si K (2014). The total operating characteristic to measure diagnostic ability for multiple thresholds, International Journal of Geographical Information Science, 28, 570-583. https://doi.org/10.1080/13658816.2013.862623
  27. Swets JA (1988). Measuring the accuracy of diagnostic systems, Science, 240, 1285-1293. https://doi.org/10.1126/science.3287615
  28. Tasche D (2008). Validation of internal rating systems and PD estimates, The Analytics of Risk Model Validation, 169-196.
  29. Vuk M and Curk T (2006). ROC curve, lift chart and calibration plot, Metodoloski Zvezki, 3, 89-108.
  30. Yoo HS and Hong CS (2011). Optimal criterion of classification accuracy measures for normal mixture, Communications for Statistical Applications and Methods, 18, 343-355. https://doi.org/10.5351/CKSS.2011.18.3.343
  31. Youden WJ (1950). Index for rating diagnostic test, Cancer, 3, 32-35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3
  32. Zweig MH and Campbell G (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine, Clinical chemistry, 39, 561-577. https://doi.org/10.1093/clinchem/39.4.561