• Title/Summary/Keyword: validation rating

Search Result 100, Processing Time 0.028 seconds

Odds curve for two classification distributions (두 분류 분포를 위한 오즈 곡선)

  • Hong, Chong Sun;Oh, Se Hyeon;Oh, Tae Gyu
    • The Korean Journal of Applied Statistics
    • /
    • v.34 no.2
    • /
    • pp.225-238
    • /
    • 2021
  • The ROC, TOC, and TROC curves, which are visually descriptive methods of exploring the performance of the binary classification model, are implemented with TP, TN, FP, FN which consist of the confusion matrix, as well as their ratios TPR, TNR, FPR, FNR. In this study, we consider two types odds and then propose an odds curve representing these odds. And show the relationship between the odds curve and ROC curve. Based on the odds curve, we propose not only two statistics that measure the discriminant power of the odds curve but also the criteria for validation ratings of the odds curve. According to the shape of the odds curves, two classification distributions can be estimated and a criterion for validation ratings can be determined. The odds curve can be meaningfully used like other visual methods, and two kinds of measures for the discriminant power can be also applied together as an alternative criterion.

Comparative study of prediction models for corporate bond rating (국내 회사채 신용 등급 예측 모형의 비교 연구)

  • Park, Hyeongkwon;Kang, Junyoung;Heo, Sungwook;Yu, Donghyeon
    • The Korean Journal of Applied Statistics
    • /
    • v.31 no.3
    • /
    • pp.367-382
    • /
    • 2018
  • Prediction models for a corporate bond rating in existing studies have been developed using various models such as linear regression, ordered logit, and random forest. Financial characteristics help build prediction models that are expected to be contained in the assigning model of the bond rating agencies. However, the ranges of bond ratings in existing studies vary from 5 to 20 and the prediction models were developed with samples in which the target companies and the observation periods are different. Thus, a simple comparison of the prediction accuracies in each study cannot determine the best prediction model. In order to conduct a fair comparison, this study has collected corporate bond ratings and financial characteristics from 2013 to 2017 and applied prediction models to them. In addition, we applied the elastic-net penalty for the linear regression, the ordered logit, and the ordered probit. Our comparison shows that data-driven variable selection using the elastic-net improves prediction accuracy in each corresponding model, and that the random forest is the most appropriate model in terms of prediction accuracy, which obtains 69.6% accuracy of the exact rating prediction on average from the 5-fold cross validation.

Identification of Uncertainty in Fitting Rating Curve with Bayesian Regression (베이지안 회귀분석을 이용한 수위-유량 관계곡선의 불확실성 분석)

  • Kim, Sang-Ug;Lee, Kil-Seong
    • Journal of Korea Water Resources Association
    • /
    • v.41 no.9
    • /
    • pp.943-958
    • /
    • 2008
  • This study employs Bayesian regression analysis for fitting discharge rating curves. The parameter estimates using the Bayesian regression analysis were compared to ordinary least square method using the t-distribution. In these comparisons, the mean values from the t-distribution and the Bayesian regression are not significantly different. However, the difference between upper and lower limits are remarkably reduced with the Bayesian regression. Therefore, from the point of view of uncertainty analysis, the Bayesian regression is more attractive than the conventional method based on a t-distribution because the data size at the site of interest is typically insufficient to estimate the parameters in rating curve. The merits and demerits of the two types of estimation methods are analyzed through the statistical simulation considering heteroscedasticity. The validation of the Bayesian regression is also performed using real stage-discharge data which were observed at 5 gauges on the Anyangcheon basin. Because the true parameters at 5 gauges are unknown, the quantitative accuracy of the Bayesian regression can not be assessed. However, it can be suggested that the uncertainty in rating curves at 5 gauges be reduced by Bayesian regression.

Development and Validation of a Scale for the Measurement of Early Childhood Teacher's Competence in Unification Education (유아교사의 통일교육역량에 대한 평가척도 개발 및 타당화 연구)

  • Jung, Dae Hyun;Kwak, Youn Mi
    • Korean Journal of Human Ecology
    • /
    • v.21 no.5
    • /
    • pp.819-835
    • /
    • 2012
  • The purpose of this study was to develop and test the validity of an assessment scale for determining the competency of early childhood teachers practicing unification education. For this purpose, an evaluation scale was constructed and then tested for reliability and validity. Participants for this study included 266 early childhood teachers in the unification education field. In order to the measure reliability and validity of this scale, Exploratory Factor Analysis and Confirmatory Factor Analysis were conducted with SPSS 18.0 and AMOS. The result of this study identified four principal factors: 1) Instruction skills, 2) Evaluation, 3) Attitude, and 4) Knowledge. The results of this study supported the scale's reliability and legitimacy as a valid instrument for the evaluation of early childhood teacher's competence in unification education.

Validation of the Penn Interactive Peer Play Scale for Korean Children (아동 또래 놀이행동 척도(PIPPS)의 국내적용을 위한 타당한 연구)

  • Choi, Hye Yeong;Shin, Hae Young
    • Korean Journal of Child Studies
    • /
    • v.29 no.3
    • /
    • pp.303-318
    • /
    • 2008
  • Participants in this study of the validity and reliability of PIPPS (Penn Interactive Peer Play Scale; Fantuzzo et al., 1998) for Korean children were 248 5-to 6-year - old children and 11 teachers. Instruments included the Peer Rating Scale(PRS; Singleton et al., 1979), Social Competence and Behavior Evaluation (SCBE; LaFreniere & Dumas, 1995), and Preschool Behavior Questionnaire (PBQ; Behar & Stringfield, 1974). The structure of PIPPS resulted in 3 factors, 'play disruption', 'play interaction', and 'play disconnection' with 30 items similar to the original PIPPS factors. Validity was evidenced by inter-correlations among sub-factors and by correlations between PIPPS and criterion measures. PIPPS scores were validated by ratings from PRS, SCBE and PBQ sub-areas scores. Cronbach's a reliability of PIPPS factors ranged from .88 to .92.

  • PDF

Estimation of Rock Mass rating(RMR) and Assessment of its Uncertainty using Conditional Simulations (조건부 모사 기법을 이용한 암반등급의 예측 및 불확실성 평가에 관한 연구)

  • Hong Chang-Woo;Jeon Seok-Won;Koo Chung-Mo
    • Tunnel and Underground Space
    • /
    • v.16 no.2 s.61
    • /
    • pp.135-145
    • /
    • 2006
  • In this study, conditional simulation was conducted to estimate rock mass rating(RMR) in unsurveyed regions. Sequential Gaussian simulation(SGS) and sequential indicator simulation(SIS) were applied for estimating RMR from the bore hole logging data. The uncertainty of SGS and SIS was verified by sample cross validation. A subset composed of 5 bore hole logging data among the original 30 bore hole logging data was set aside as test data. The remainder was training data. The quality of SGS and SIS estimation on the testing data reflects how well it would perform in an unsupervised setting. SGS and SIS were useful stochastic methods to estimate the spatial distribution of rock mass classes correctly and assess the uncertainty of estimation quantitatively. The result of conditional simulation can offer useful information of rock mass classes such as RMR in unsurveyed regions.

Grading System of Movie Review through the Use of An Appraisal Dictionary and Computation of Semantic Segments (감정어휘 평가사전과 의미마디 연산을 이용한 영화평 등급화 시스템)

  • Ko, Min-Su;Shin, Hyo-Pil
    • Korean Journal of Cognitive Science
    • /
    • v.21 no.4
    • /
    • pp.669-696
    • /
    • 2010
  • Assuming that the whole meaning of a document is a composition of the meanings of each part, this paper proposes to study the automatic grading of movie reviews which contain sentimental expressions. This will be accomplished by calculating the values of semantic segments and performing data classification for each review. The ARSSA(The Automatic Rating System for Sentiment analysis using an Appraisal dictionary) system is an effort to model decision making processes in a manner similar to that of the human mind. This aims to resolve the discontinuity between the numerical ranking and textual rationalization present in the binary structure of the current review rating system: {rate: review}. This model can be realized by performing analysis on the abstract menas extracted from each review. The performance of this system was experimentally calculated by performing a 10-fold Cross-Validation test of 1000 reviews obtained from the Naver Movie site. The system achieved an 85% F1 Score when compared to predefined values using a predefined appraisal dictionary.

  • PDF

Consumers'′ Evaluation toward Retail Salespeople Attributes : Scale Development, Validation, and Some Related Variables (소비자가 지각하는 백화점 의류판매원의 평가속성 : 측정도구 개발 및 관련변인)

  • 진병호;홍병숙
    • Journal of Distribution Research
    • /
    • v.4 no.3
    • /
    • pp.65-86
    • /
    • 2000
  • From the perspective of relationship marketing, salespeople acts important role by creating values and developing, establishing long-term relationship with customers. This study is designed to suggest validated measure of evaluating retail salespeople in department store setting in an effort to facilitate active future studies. For this purpose, this study investigated attributes of retail salespeople based on series of personal interviews and literature reviews, and validated its measurement via exploratory and confirmatory factor analysis. In addition, this study explored consumers' rating of the importance of attributes related to some selected consumer variables (consumer knowledge, product involvement, and demographic variables). The findings of this study revealed five dimensions of retail salespeople attributes: Service Mind, Sales Efforts, Product Knowledge, Comfortable Impression and Sale Inducing Skill. Customers of department stores put importance of salespeople attributes in this order. Some dimensions of consumer knowledge and product involvement do affect consumers' rating of retail salespeople attributes. Theoretical and managerial implications were suggested based on empirical results.

  • PDF

Remanufacturing Process Design for Automotive Alternator (자동차 교류발전기의 재제조 프로세스 설계)

  • Roslan, Liyana;Azmi, Nurul Ain;Jung, Won
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.34 no.4
    • /
    • pp.179-188
    • /
    • 2011
  • This paper outlines a systematic guideline for remanufacturing process using the Failure Mode and Effect Analysis (FMEA) method in order to estimate the reliability and quality of the remanufactured alternator. The method is just a tool to help, but the remanufacturer must determine the optimal remanufacturing process and specific inspection and production that will turn the alternator as-good-as new and place the product into the market with reliability and quality equal to a new product. FMEA is a method that is widely used in industry and has shown its value and effectiveness in the above remanufacturing case study. Actions taken often result in a lower severity, occurrence or detection rating. Redesign may result in lower severity and occurrence ratings while inserting validation controls and maintenance can reduce the detection rating. The revised ratings are recorded with the originals on the FMEA template form. After these corrective actions and revisions have been established, evaluation of the ranks can be repeated, until the redesign and control parameters comply with safety standards.

A Study on Developing the Evaluation Items for Estimating the Digitization Level of Libraries (도서관의 디지털화 수준 평가항목 개발에 관한 연구)

  • Noh, Younghee
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.50 no.2
    • /
    • pp.47-75
    • /
    • 2016
  • This study was conducted to develop items for evaluating the level of the digitization of libraries. For this purpose, it analyzed the literature related to the digital library, underwent a convergence process of 10 experts, and finally derived an axis of 13 different dimensions comparing the digitization of libraries. The axis is composed of acquisitions, collections (physical and online collections), classification and cataloging, circulation service, reference, user service, SNS service, the library's organization and staff, device providing service, next-generation service, and status of our library. This study conducted a survey of librarians to secure the validation of the primarily derived evaluation items regarding libraries' digitization. As a result, the average rating of the traditional evaluation items was 3.82, and the average rating of the digital evaluation items was 4.08. Therefore, it can be said that the results of this research to evaluate the digitized level of libraries have a certain degree of validity.