Journal of the Korean Data and Information Science Society

The Korean Data and Information Science Society (한국데이터정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Two optimal threshold criteria for ROC analysis

  • Cho, Min Ho (Department of Statistics, Sungkyunkwan University) ;
  • Hong, Chong Sun (Department of Statistics, Sungkyunkwan University)
  • Received : 2014.12.08
  • Accepted : 2015.01.10
  • Published : 2015.01.31
Download PDF
⟨ Previous Next ⟩

Abstract

Among many optimal threshold criteria from ROC curve, the closest-to-(0,1) and amended closest-to-(0,1) criteria are considered. An ROC curve that passes close to the (0,1) point indicates that two models are well classified. In this case, the ROC curve is located far from the (1,0) point. Hence we propose two criteria: the farthest-to-(1,0) and amended farthest-to-(1,0) criteria. These criteria are found to have a relationship with the KolmogorovSmirnov statistic as well as some optimal threshold criteria. Moreover, we derive that a definition for the proposed criteria with more than two dimensions and with relations to multi-dimensional optimal threshold criteria.

Keywords

References

  1. Brasil, P. (2010). DiagnosisMed: Diagnostic test accuracy evaluation for medical professionals. R package version 0.2.3.
  2. Connell, F. A. and Koepsell, T. D. (1985). Measures of gain in certainty from a diagnostic test. American Journal of Epidiology, 121, 744-753. https://doi.org/10.1093/aje/121.5.744
  3. Engelmann, B., Hayden, E. and Tasche, D. (2003). Measuring the discriminative power of rating systems. Risk, 82-86.
  4. Fawcett, T. (2003). ROC graphs: notes and practical considerations for data mining researchers, HP Labs Tech Report HPL-2003-4, CA, USA.
  5. Hong, C. S. (2009). Optimal threshold from ROC and CAP curves. Communications in Statistics - Simulation and Computation, 38, 2060-2072. https://doi.org/10.1080/03610910903243703
  6. Hong, C. S. and Joo, J. S. (2010). Optimal thresholds from non-normal mixture. The Korean Journal of Applied Statistics, 23, 943-953. https://doi.org/10.5351/KJAS.2010.23.5.943
  7. Hong, C. S., Joo, J. S. and Choi, J. S. (2010). Optimal thresholds from mixture distributions. The Korean Journal of Applied Statistics, 23, 13-28. https://doi.org/10.5351/KJAS.2010.23.1.013
  8. Hong, C. S. and Jung, E. S. (2013). Optimal thresholds criteria for ROC surfaces. Journal of the Korean Data & Information Science Society, 24, 1489-1496. https://doi.org/10.7465/jkdi.2013.24.6.1489
  9. Hong, C. S. and Wu, Z. Q. (2014). Alternative accuracy for multiple ROC analysis. Journal of the Korean Data & Information Science Society, 25, 1521-1530. https://doi.org/10.7465/jkdi.2014.25.6.1521
  10. Krzanowshi, W. J. and Hand, D. J. (2009). ROC curves for continuous data, Chapman and Hall, London.
  11. Li, J. and Fine, J. P. (2008). ROC analysis with multiple classes and multiple tests: Methodology and its application in microarray studies. Biostatistics, 9, 566-576. https://doi.org/10.1093/biostatistics/kxm050
  12. Perkins, N. J. and Schisterman, E. F. (2006). The inconsistency of optimal cutpoints obtained using two criteria based on based on the receiver operating characteristic curve. American Journal of Epidiology, 163, 670-675. https://doi.org/10.1093/aje/kwj063
  13. Provost, F. and Fawcett, T. (2001). Robust classification for imprecise environments. Machine Learning, 42, 203-231. https://doi.org/10.1023/A:1007601015854
  14. Sobehart, J. R. and Keenan, S. C. (2001). Measuring default accurately. Credit Risk Special Report, Risk, 14, 31-33.
  15. Velez, D. R., White, B. C., Motsinger, A. A., Bush, W. S., Ritchie, M. D., Williams, S. M. and Moore, J. H. (2007). A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction. Genetic Epidiology, 31, 306-315. https://doi.org/10.1002/gepi.20211
  16. Youden, W. J. (1950). Index for rating diagnostic test. Cancer, 3, 32-35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3
  17. Zou, K. H., O'Malley, A. J. and Mauri, L. (2007). Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation, 115, 654-657. https://doi.org/10.1161/CIRCULATIONAHA.105.594929

Cited by

  1. Parameter estimation of linear function using VUS and HUM maximization vol.26, pp.6, 2015, https://doi.org/10.7465/jkdi.2015.26.6.1305
  2. TPR-TNR plot for confusion matrix vol.28, pp.2, 2021, https://doi.org/10.29220/csam.2021.28.2.161