Browse > Article
http://dx.doi.org/10.5351/KJAS.2021.34.2.225

Odds curve for two classification distributions  

Hong, Chong Sun (Department of Statistics, Sungkyunkwan University)
Oh, Se Hyeon (Department of Statistics, Sungkyunkwan University)
Oh, Tae Gyu (Department of Statistics, Sungkyunkwan University)
Publication Information
The Korean Journal of Applied Statistics / v.34, no.2, 2021 , pp. 225-238 More about this Journal
Abstract
The ROC, TOC, and TROC curves, which are visually descriptive methods of exploring the performance of the binary classification model, are implemented with TP, TN, FP, FN which consist of the confusion matrix, as well as their ratios TPR, TNR, FPR, FNR. In this study, we consider two types odds and then propose an odds curve representing these odds. And show the relationship between the odds curve and ROC curve. Based on the odds curve, we propose not only two statistics that measure the discriminant power of the odds curve but also the criteria for validation ratings of the odds curve. According to the shape of the odds curves, two classification distributions can be estimated and a criterion for validation ratings can be determined. The odds curve can be meaningfully used like other visual methods, and two kinds of measures for the discriminant power can be also applied together as an alternative criterion.
Keywords
confusion matrix; odds; sensitivity; specificity; validation rating;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Cho, M. H. and Hong, C. S. (2015). Two optimal threshold criteria for ROC analysis, Journal of the Korean Data and Information Science Society, 26, 255-260.   DOI
2 Egan, J. P. and Egan, J. P. (1975). Signal Detection Theory and ROC-Analysis, Academic Press.
3 Engelmann, B., Hayden, E., and Tasche, D. (2003). Testing rating accuracy, Risk, 16, 82-86.
4 Fawcett, T. (2004). ROC graphs: Notes and practical considerations for researchers, Machine Learning, 31, 1-38.
5 Fawcett, T. (2006). An introduction to ROC analysis, Pattern Recognition Letters, 27, 861-874.   DOI
6 Green, D. M. and Swets, J. A. (1966). Signal Detection Theory and Psychophysics (Vol. 1), Wiley, New York.
7 Hong, C. S. and Wu, Z. Q. (2014). Alternative accuracy for multiple ROC analysis, Journal of the Korean Data & Information Science Society, 25, 1521-1530.   DOI
8 Hong, C. S. (2021). Confusion plot for the confusion matrix, The Korean Data & Information Science Society, 32, 427-437.   DOI
9 Hong, C. S. and Jang, D. H. (2020). Partial AUC using the sensitivity and specificity lines, The Korean Journal of Applied Statistics, 33, 541-553.   DOI
10 Hong, C. S., Joo, J. S., and Choi, J. S. (2010). Optimal thresholds from mixture distributions, The Korean Journal of Applied Statistics, 23, 13-28.   DOI
11 Hong, C. S., Jung, E. S., and Jung, D. G. (2013). Standard criterion of VUS for ROC surface, The Korean Journal of Applied Statistics, 26, 977-985.   DOI
12 Hong, C. S., Jung, M. S., and Shin, H. S. (2019a). Partial VUS and optimal thresholds, The Korean Data & Information Science Society, 30, 895-905.   DOI
13 Hong, C. S., Jung, M. S., and Shin, H. S. (2019b). Three-way partial VUS, The Korean Data & Information Science Society, 30, 445-454.   DOI
14 Hong, C. S. and Lee, S. J. (2018). TROC curve and accuracy measures, The Korean Data & Information Science Society, 29, 861-872.   DOI
15 Joseph, M. P. (2005). A PD validation framework for Basel II internal ratings-based systems, Credit Scoring and Credit Control IV.
16 Hong, C. S. and Na, H. R. (2020). Maximum absolute determinant of the confusion matrix, The Korean Data & Information Science Society, 31, 937-950.   DOI
17 Hsieh, F. and Turnbull, B. W. (1996). Nonparametric and semiparametric estimation of the receiver operating characteristic curve, Annals of Statistics, 24, 25-40.
18 Jeni, L. A., Cohn, J. F., and De La Torre, F. (2013). Facing imbalanced data-recommendations for the use of performance metrics. In 2013 Humaine association conference on affective computing and intelligent interaction (pp. 245-251). IEEE.
19 Swets, J. A. (1988). Measuring the accuracy of diagnostic systems, Science, 240, 1285-1293.   DOI
20 Stein, R. M. (2005). The relationship between default prediction and lending profits: Integrating ROC analysis and loan pricing, Journal of Banking & Finance, 29, 1213-1236.   DOI
21 Tasche, D. (2008). Validation of internal rating systems and PD estimates. In The Analytics of Risk Model Validation, Academic Press, 169-196.
22 Vuk, M. and Curk, T. (2006). ROC curve, lift chart and calibration plot, Metodoloski Zvezki, 3, 89.
23 Zweig, M. H. and Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine, Clinical Chemistry, 39, 561-577.   DOI
24 Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition, 30, 1145-1159.   DOI
25 Metz, C. E. (1978). Basic principles of ROC analysis. In Seminars in nuclear medicine (Vol. 8, No. 4, pp. 283-298). WB Saunders.
26 Metz, C. E. and Kronman, H. B. (1980). Statistical significance tests for binormal ROC curves, Journal of Mathematical Psychology, 22, 218-243.   DOI
27 Pepe, M. S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press, Oxford.
28 Altman, D. G. and Bland, J. M. (1994). Diagnostic tests. 1: Sensitivity and specificity, BMJ: British Medical Journal, 308, 1552.   DOI
29 Bamber, D. (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph, Journal of Mathematical Psychology, 12, 387-415.   DOI
30 Centor, R. M. (1991). Signal detectability: the use of ROC curves and their analyses, Medical Decision Making, 11, 102-106.   DOI
31 Sonego, P., Kocsor, A., and Pongor, S. (2008). ROC analysis: applications to the classification of biological sequences and 3D structures, Briefings in Bioinformatics, 9, 198-209.   DOI
32 Hanley, J. A. and McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143, 29-36.   DOI
33 Provost, F. and Fawcett, T. (1997). Analysis and visualization of classifier performance with nonuniform class and cost distributions. In Proceedings of AAAI-97 Workshop on AI Approaches to Fraud Detection & Risk Management, 57-63.
34 Pepe, M. S., Longton, G., Anderson, G. L., and Schummer, M. (2003). Selecting differentially expressed genes from microarray experiments, Biometrics, 59, 133-142.   DOI
35 Pontius Jr, R. G. and Si, K. (2014). The total operating characteristic to measure diagnostic ability for multiple thresholds, International Journal of Geographical Information Science, 28, 570-583.   DOI
36 Provost, F. and Fawcett, T. (2001). Robust classification for imprecise environments, Machine Learning, 42, 203-231.   DOI