Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

TPR-TNR plot for confusion matrix

  • Hong, Chong Sun (Department of Statistics, Sungkyunkwan University) ;
  • Oh, Tae Gyu (Department of Statistics, Sungkyunkwan University)
  • Received : 2020.11.04
  • Accepted : 2021.02.02
  • Published : 2021.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

The two-dimensional confusion matrix used in credit assessment, biostatistics, and many other fields consists of true positive, true negative, false positive, and false negative. Their rates, such as the true positive rate (TPR), true negative rate (TNR), false positive rate, and false negative rate, can be applied to measure its accuracy. In this study, we propose the TPR-TNR plot, a graphical method that can geometrically describe and explain these rates based on the confusion matrix. The proposed TPR-TNR plot consists of two right-angled triangles. We obtain that the TPR and TNR describe the acute angles of right-angled triangles in the plot. These acute angles can be used to determine optimal thresholds corresponding to lots of accuracy measures.

Keywords

References

  1. Altman DG and Bland JM (1994). Diagnostic tests. 1: Sensitivity and specificity, BMJ: British Medical Journal, 308, 1552. https://doi.org/10.1136/bmj.308.6943.1552
  2. Bamber D (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph, Journal of Mathematical Psychology, 12, 387-415. https://doi.org/10.1016/0022-2496(75)90001-2
  3. Brasil P (2010). Diagnostic test accuracy evaluation for medical professionals, Package DiagnosisMed in R.
  4. Centor RM (1991). Signal detectability: the use of ROC curves and their analyses, Medical Decision Making, 11, 102-106. https://doi.org/10.1177/0272989X9101100205
  5. Cho MH and Hong CS (2015). Two optimal threshold criteria for ROC analysis, The Korean Data & Information Science Society, 26, 255-260. https://doi.org/10.7465/jkdi.2015.26.1.255
  6. Connell FA and Koepsell TD (1985). Measures of gain in certainty from a diagnostic test, American Journal of Epidemiology, 121, 744-753. https://doi.org/10.1093/aje/121.5.744
  7. Egan JP and Egan JP (1975). Signal Detection Theory and ROC-Analysis, Academic Press, New York.
  8. Engelmann B, Hayden E, and Tasche D (2003). Testing rating accuracy, Risk, 16, 82-86.
  9. Fawcett T (2004). ROC graphs: Notes and practical considerations for researchers, Machine Learning, 31, 1-38.
  10. Fawcett T (2006). An introduction to ROC analysis, Pattern Recognition Letters, 27, 861-874. https://doi.org/10.1016/j.patrec.2005.10.010
  11. Fluss R, Faraggi D, and Reiser B (2005). Estimation of the Youden Index and its associated cutoff point, Biometrical Journal: Journal of Mathematical Methods in Biosciences, 47, 458-472. https://doi.org/10.1002/bimj.200410135
  12. Green DM and Swets JA (1966). Signal Detection Theory and Psychophysics (Vol. 1), Wiley, New York.
  13. Hanley JA and McNeil BJ (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143, 29-36. https://doi.org/10.1148/radiology.143.1.7063747
  14. Hong CS and Choi SY (2020). Positive and negative predictive values by the TOC curve, Communications for Statistical Applications and Methods, 27, 211-224. https://doi.org/10.29220/CSAM.2020.27.2.211
  15. Hong CS, Joo JS, and Choi JS (2010). Optimal thresholds from mixture distributions, The Korean Journal of Applied Statistics, 23, 13-28. https://doi.org/10.5351/KJAS.2010.23.1.013
  16. Hong CS and Lee SJ (2018). TROC curve and accuracy measures, The Korean Data & Information Science Society, 29, 861-872. https://doi.org/10.7465/jkdi.2018.29.4.861
  17. Hong CS and Yoo HS (2011). Optimal criterion of classification accuracy measures for normal mixture, Communications for Statistical Applications and Methods, 18, 343-355. https://doi.org/10.5351/CKSS.2011.18.3.343
  18. Hsieh F and Turnbull BW (1996). Nonparametric and semiparametric estimation of the receiver operating characteristic curve, The Annals of Statistics, 24, 25-40.
  19. Krzanowski WJ and Hand DJ (2009). ROC Curves for Continuous Data, CRC Press, Boston.
  20. Liu X (2012). Classification accuracy and cut point selection, Statistics in Medicine, 31, 2676-2686. https://doi.org/10.1002/sim.4509
  21. Metz CE and Kronman HB (1980). Statistical significance tests for binormal ROC curves, Journal of Mathematical Psychology, 22, 218-243. https://doi.org/10.1016/0022-2496(80)90020-6
  22. Moses LE, Shapiro D, and Littenberg B (1993). Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations, Statistics in Medicine, 12, 1293-1316. https://doi.org/10.1002/sim.4780121403
  23. Oehr P and Ecke T (2020). Establishment and characterization of an empirical biomarker SS/PV-ROC plot using results of the UBC® rapid test in bladder cancer, Entropy, 22, 729. https://doi.org/10.3390/e22070729
  24. Pepe MS (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press, Oxford.
  25. Perkins NJ and Schisterman EF (2005). The Youden Index and the optimal cut-point corrected for measurement error, Biometrical Journal: Journal of Mathematical Methods in Biosciences, 47, 428-441. https://doi.org/10.1002/bimj.200410133
  26. Perkins NJ and Schisterman EF (2006). The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve, American Journal of Epidemiology, 163, 670-675. https://doi.org/10.1093/aje/kwj063
  27. Pontius Jr RG and Si K (2014). The total operating characteristic to measure diagnostic ability for multiple thresholds, International Journal of Geographical Information Science, 28, 570-583. https://doi.org/10.1080/13658816.2013.862623
  28. Provost F and Fawcett T (1997). Analysis and visualization of classifier performance with nonuniform class and cost distributions. In Proceedings of AAAI-97 Workshop on AI Approaches to Fraud Detection & Risk Management, 57-63.
  29. Provost F and Fawcett T (2001). Robust classification for imprecise environments, Machine Learning, 42, 203-231. https://doi.org/10.1023/A:1007601015854
  30. Sonego P, Kocsor A, and Pongor S (2008). ROC analysis: applications to the classification of biological sequences and 3D structures, Briefings in Bioinformatics, 9, 198-209. https://doi.org/10.1093/bib/bbm064
  31. Stein RM (2005). The relationship between default prediction and lending profits: Integrating ROC analysis and loan pricing, Journal of Banking & Finance, 29, 1213-1236. https://doi.org/10.1016/j.jbankfin.2004.04.008
  32. Swets JA (1988). Measuring the accuracy of diagnostic systems, Science, 240, 1285-1293. https://doi.org/10.1126/science.3287615
  33. Tasche D (2008). Validation of internal rating systems and PD estimates. In The Analytics of Risk Model Validation (PP. 169-196), Academic Press.
  34. Vuk M and Curk T (2006). ROC curve, lift chart and calibration plot, Metodoloski Zvezki, 3, 89-108.
  35. Youden WJ (1950). Index for rating diagnostic tests, Cancer, 3, 32-35. https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3
  36. Zweig MH and Campbell G (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine, Clinical Chemistry, 39, 561-577. https://doi.org/10.1093/clinchem/39.4.561