• Journal of the Korean Data and Information Science Society
• /
• v.26 no.1
• /
• pp.255-260
• /
• 2015

Among many optimal threshold criteria from ROC curve, the closest-to-(0,1) and amended closest-to-(0,1) criteria are considered. An ROC curve that passes close to the (0,1) point indicates that two models are well classified. In this case, the ROC curve is located far from the (1,0) point. Hence we propose two criteria: the farthest-to-(1,0) and amended farthest-to-(1,0) criteria. These criteria are found to have a relationship with the KolmogorovSmirnov statistic as well as some optimal threshold criteria. Moreover, we derive that a definition for the proposed criteria with more than two dimensions and with relations to multi-dimensional optimal threshold criteria.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.5 no.3
• /
• pp.263-267
• /
• 2005

Building classifiers for financial real-world classification problems is often plagued by severely overlapping and highly skewed class distribution. New performance measures such as receiver operating characteristic (ROC) curve and area under ROC curve (AUC) have been recently introduced in evaluating and building classifiers for those kind of problems. They are, however, in-effective to evaluation of classifier's discrimination performance in a particular class of the classification problems that interests lie in only a local operating range of the classifier, In this paper, a new method is proposed that enables us to directly improve classifier's discrimination performance at a desired local operating range by defining and optimizing a partial area under ROC curve or domain-specific curve, which is difficult to achieve with conventional classification accuracy based learning methods. The effectiveness of the proposed approach is demonstrated in terms of fraud detection capability in a real-world fraud detection problem compared with the MSE-based approach.

• The Korean Journal of Applied Statistics
• /
• v.24 no.1
• /
• pp.185-191
• /
• 2011

In the credit evaluation study with the assumption of mixture distributions, the ROC curve is a useful method to explore the discriminatory power of default and non-default borrowers. The AROC curve is an adjusted ROC curve that can be identified with the corresponding score and is mathematically analyzed in this work. We obtain patterns of this curve by applying normal distributions. Moreover, the relationship between the AROC curve and many classification accuracy statistics are explored to find the optimal threshold. In the case of equivalent variances of two distributions, we obtain that the local minimum of the AROC curve is estimated at the optimal threshold to maximize certain classification accuracies.

• Communications for Statistical Applications and Methods
• /
• v.26 no.2
• /
• pp.205-216
• /
• 2019

Diagnostic tests in medical fields detect or diagnose a disease with results measured by continuous or discrete ordinal data. The performance of a diagnostic test is summarized using the receiver operating characteristic (ROC) curve and the area under the curve (AUC). The diagnostic test is considered clinically useful if the outcomes in actually-positive cases are higher than actually-negative cases and the ROC curve is concave. In this study, we apply the stochastic ordering method in a Bayesian hierarchical model to estimate the proper ROC curve and AUC when the diagnostic test results are measured in discrete ordinal data. We compare the conventional binormal model and binormal model under stochastic ordering. The simulation results and real data analysis for breast cancer indicate that the binormal model under stochastic ordering can be used to estimate the proper ROC curve with a small bias even though the sample sizes were small or the sample size of actually-negative cases varied from actually-positive cases. Therefore, it is appropriate to consider the binormal model under stochastic ordering in the presence of large differences for a sample size between actually-negative and actually-positive groups.

• The Korean Journal of Applied Statistics
• /
• v.24 no.2
• /
• pp.269-278
• /
• 2011

There are many researches that have considered the distribution functions and appropriate covariates corresponding to the scores in order to improve the accuracy of a diagnostic test, including the ROC curve that is represented with the relations of the sensitivity and the specificity. The ROC analysis was used by the regression model including some covariates under the assumptions that its distribution function is known or estimable. In this work, we consider a general situation that both the distribution function and the elects of covariates are unknown. For the ROC analysis, the mixtures of normal distributions are used to estimate the distribution function fitted to the credit evaluation data that is consisted of the score random variable and two sub-populations of parameters. The AUC measure is explored to compare with the nonparametric and empirical ROC curve. We conclude that the method using normal mixtures is fitted to the classical one better than other methods.

• Analytical Science and Technology
• /
• v.34 no.2
• /
• pp.78-86
• /
• 2021

The objective of this study was to investigate urinary cut-off concentrations of quetiapine and risperidone for distinction between normal and abnormal/non-takers who were being placed on probation. Liquid chromatography-tandem mass spectrometric (LC-MS/MS) method was employed for determination of antipsychotic drugs in urine from mentally disordered probationers. The optimal cut-off values of antipsychotic drugs were calculated using receiver operating characteristic (ROC) curve analysis. The sensitivity and specificity of the method for the detection of antipsychotic drugs in urine were subsequently evaluated. The area under the ROC curve (AUC) was 0.927 for norquetiapine and 0.791 for 9-hydroxyrisperidone, respectively. These antipsychotic drugs are classified readily in the ROC curve analysis. The cut-off values for distinguishing regular and irregular/non-takers were 39.1 ng/mL for norquetiapine and 67.9 ng/mL for 9-hydroxyrisperidone, respectively. The results of this study suggest the cut-off values of quetiapine and risperidone were highly useful to distinguish regular takers from irregular/non-takers.

• Journal of the Korean Society of Radiology
• /
• v.10 no.8
• /
• pp.645-652
• /
• 2016

In this study, based on the analysis of texture feature values of statistical properties. And we examined the normal and the applicability of the computer-aided diagnosis of cerebral infarction in the brain computed tomography images. The experiment was analyzed to evaluate the ROC curve recognition rate of disease using six parameters representing the feature values of the texture. As a result, it showed average mean 88%, variance 92%, relative smoothness 94%, uniformity of 88%, a high disease recognition rate of entropy 84%. However, it showed a slightly lower disease recognition rate and 58% for skewness. In the analysis using ROC curve, the area under the curve for each parameter indicates 0.886 (p = 0.0001) or more, resulted in a meaningful recognition of the disease. Further, to determine the cut-off values for each parameter are determined to be the prediction of disease through the computer-aided diagnosis.

• Communications for Statistical Applications and Methods
• /
• v.19 no.4
• /
• pp.629-638
• /
• 2012

We propose some methods to obtain optimal thresholds and classification functions by using various cutoff criterion based on the bivariate ROC curve that represents bivariate cumulative distribution functions. The false positive rate and false negative rate are calculated with these classification functions for bivariate normal distributions.

• Journal of Veterinary Clinics
• /
• v.19 no.3
• /
• pp.312-315
• /
• 2002

Diagnostic tests often require the determination of cut-off values that discriminate uninfected from infected individuals. The receiver operating characteristic (ROC) curve has been frequently used to attain this purpose and gives a representation of diagnostic accuracy (sensitivity and specificity) of a prediction model when varying the cut-point of a decision rule on a whole spectrum. We have written and tested a visual basic application program in EXCEL for maximum likelihood estimation of a binormal ROC curve, which also computes univariate statistics of a diagnostic test employed. Examples applying for computed tomographic images in radiology and methicillin-resistant Staphylococcus aureus research are given to illustrate this approach. This stand-alone module is available from the first author on request.

• Journal of radiological science and technology
• /
• v.32 no.2
• /
• pp.147-151
• /
• 2009

In general, the discrete confidence judgments that use five-step assessment method have been used to assess the medical images by ROC. TPF or FPF can be computed easily with this independent reading test. However, during experiments, it happens frequently that adequate distribution for observers is required to smoothly estimate the ROC curve. In addition, data becomes invalid for distribution of the created categories. To solve such problems or to apply the ROC interpretation to data that is not obtained from the experimental observation, the continuous confidence judgements (CCJ) has been proposed, which implements ROC interpretation using continuously-distributed experimental results without category classification has been used. As the use of CCJ to assess medical images was barely reported in Korea, we applied it to the assessment of chest digital images in this study. The results showed that a smooth ROC curve was obtained conveniently by the commercialized program and the characteristic value was measured easily. Therefore, it is recommended that this method can be applied to the assessment of digital medical images.