Browse > Article
http://dx.doi.org/10.7465/jkdi.2016.27.1.33

Standardized polytomous discrimination index using concordance  

Choi, Jin Soo (Department of Statistics, Sungkyunkwan University)
Hong, Chong Sun (Department of Statistics, Sungkyunkwan University)
Publication Information
Journal of the Korean Data and Information Science Society / v.27, no.1, 2016 , pp. 33-44 More about this Journal
Abstract
There are many situations that the outcome for clinical decision and credit assessment should be predicted more than two categories. Five kinds of statistics which are used the concordance are proposed and used for these polytomous problems. However, these statistics are defined without exact distinction of categories, so that we have difficulty to use both the pair and set approaches and it is hard to understand the meanings of these statistics. Hence, it is not possible to compare and analyze them. In this paper, the polytomous confusion matrix is standardized and the concordance statistic can be represented based on the confusion matrix. The five kinds of statistics by using the concordance are defined. With the methods proposed in this paper, we could not only explain their meanings but also compare and analyze these statistics. Based on various data sets, properties of these five statistics are explored and explained.
Keywords
Concordance; credit rating; discrimination index; evaluation data; polytomous;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 Ferri, C., Hernandez-Orallo, J. and Salido, M. A. (2003). Volume under the ROC surface for multi-class problems, Exact computation and evaluation of approximations, Technical report, Univ. Politecnica de Valencia, Spain.
2 Hand, D. J. and Till, R. J. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problem. Machine Learning, 45, 171-186.   DOI
3 Harrell, F. E. Jr. (2001). Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis, Springer, New York.
4 Harrell, F. E. Jr., Lee, K. L. and Mark, D. B. (1996). Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in medicine, 15, 361-387.   DOI
5 Hilden, J. (1991). The area under the ROC curve and its competitors. Medical Decision Making, 11, 95-101.   DOI
6 Hong, C. S. and Cho, M. H. (2015a). VUS and HUM represented with Mann-Whitney statistic. Communications for Statistical Applications and Methods, 22, 223-232.   DOI
7 Hong, C. S. and Cho, M. H. (2015b). Test statistics for volume under the ROC surface and hypervolume under the ROC manifold. Communications for Statistical Applications and Methods, 22, 377-387.   DOI
8 Hong, C. S. and Choi, J. M. (2008). Validation comparison of credit rating models using Box-Cox transformation. Journal of the Korean Data & Information Science Society, 19, 789-801.
9 Hong, C. S. and Kim, J. H. (2009). Nonparametric homogeneity tests of two distributions for credit rating model validation. Journal of the Korean Data & Information Science Society, 20, 261-272.
10 Mossman, D. (1999). Three-way ROCs. Medical Decision Making, 19, 78-89.   DOI
11 Nakas, C. T. and Alonzo, T. A. (2007). ROC graphs for assessing the ability of a diagnostic marker to detect three disease classes with an umbrella ordering. Biomertics, 63, 603-609.   DOI
12 Nakas, C. T. and Yiannoutsos, C. T. (2004). Ordered multiple-class ROC analysis with continuous measurements. Statistics in Medicine, 23, 2437-2449.
13 Provost, F. and Domingos, P. (2003). Tree induction for probability-based ranking. Machine Learning, 52, 199-215.   DOI
14 Obuchowski, N. A., Goske, M. J. and Applegate, K. E. (2001). Assessing physicians' accuracy in diagnosing paediatric patients with acute abdominal pain: Measuring accuracy for multiple diseases. Statistics in Medicine, 20, 3261-3278.   DOI
15 Obuchowski, N. A. (2005). Estimating and comparing diagnostic tests' accuracy when the gold standard is not binary. Academic Radiology, 12, 1198-1204.   DOI
16 Pencina, M. J., D'Agostino, R. B., D'Agostino, R. B. Jr. and Vasan, R. S. (2008). Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Statistics in Medicine, 27, 157-172.   DOI
17 Van Calster, B. and Van Huffel, S. (2010). Integrated discrimination improvement and probability-sensitive AUC variants. Statistics in Medicine, 29, 318-319.   DOI
18 Van Calster, B., Van Belle, V., Vergouwe, Y. and Steyerberg, E. W. (2012). Discrimination ability of prediction models for ordinal outcomes: Relationship between existing measures and a new measure. Biometrical Journal, 54, 674-685.   DOI
19 Van Calster, B., Van Belle, V., Vergouwe, Y., Timmerman, D., Van Huffel, S. and Steyerberg, E. W. (2012). Extending the c-statistic to nominal polytomous outcomes: The Polytomous Discrimination Index. Statistics in Medicine, 31, 2610-2626.   DOI
20 Van Calster, B., Vergouwe, Y., Van Belle, V., Looman, C., Timmerman, D. and Steyerberg, E. W. (2012). Assessing the discriminative ability of risk models for more than two outcome categories: A perspective. European Journal of Epidemiology, 27, 761-770.   DOI