• Title/Summary/Keyword: Test Validation

Search Result 1,777, Processing Time 0.033 seconds

Some Issues on Criterion for Kolmogorov-Smirnov Test in Credit Rating Model Validation (신용평가모형에서 콜모고로프-스미르노프 검정기준의 문제점)

  • Park, Yong-Seok;Hong, Chong-Sun
    • Communications for Statistical Applications and Methods
    • /
    • v.15 no.6
    • /
    • pp.1013-1026
    • /
    • 2008
  • Kolmogorov-Smirnov(K-S) statistic has been widely used for the model validation of credit rating models. Validation criteria for the K-S statistic is empirically used at the levels of 0.3 or 0.4 which are much larger than the critical values of K-S test statistic. We examine whether these criteria are reasonable and appropriate through the simulations according to various sample sizes, type II error rates, and the ratio of bads among data. The simulation results say that the currently used validation criteria are too lower than values of K-S statistics obtained from any credit rating models in Korea, so that any credit rating models have good discriminatory power. In this work, alternative criteria of K-S statistic are proposed as critical levels under realistic situations of credit rating models.

A Study on Time Series Cross-Validation Techniques for Enhancing the Accuracy of Reservoir Water Level Prediction Using Automated Machine Learning TPOT (자동기계학습 TPOT 기반 저수위 예측 정확도 향상을 위한 시계열 교차검증 기법 연구)

  • Bae, Joo-Hyun;Park, Woon-Ji;Lee, Seoro;Park, Tae-Seon;Park, Sang-Bin;Kim, Jonggun;Lim, Kyoung-Jae
    • Journal of The Korean Society of Agricultural Engineers
    • /
    • v.66 no.1
    • /
    • pp.1-13
    • /
    • 2024
  • This study assessed the efficacy of improving the accuracy of reservoir water level prediction models by employing automated machine learning models and efficient cross-validation methods for time-series data. Considering the inherent complexity and non-linearity of time-series data related to reservoir water levels, we proposed an optimized approach for model selection and training. The performance of twelve models was evaluated for the Obong Reservoir in Gangneung, Gangwon Province, using the TPOT (Tree-based Pipeline Optimization Tool) and four cross-validation methods, which led to the determination of the optimal pipeline model. The pipeline model consisting of Extra Tree, Stacking Ridge Regression, and Simple Ridge Regression showed outstanding predictive performance for both training and test data, with an R2 (Coefficient of determination) and NSE (Nash-Sutcliffe Efficiency) exceeding 0.93. On the other hand, for predictions of water levels 12 hours later, the pipeline model selected through time-series split cross-validation accurately captured the change pattern of time-series water level data during the test period, with an NSE exceeding 0.99. The methodology proposed in this study is expected to greatly contribute to the efficient generation of reservoir water level predictions in regions with high rainfall variability.

Design of Validation System for a Crypto-Algorithm Implementation (암호 알고리즘 구현 적합성 평가 시스템 설계)

  • Ha, Kyeoung-Ju;Seo, Chang-Ho;Kim, Dae-Youb
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.39B no.4
    • /
    • pp.242-250
    • /
    • 2014
  • Conventional researches of standard tool validating cryptographic algorithm have been studied for the internet environment, for the mobile internet. It is important to develop the validation tool for establishment of interoperability and convenience of users in the information systems. Therefore, this paper presents the validation tool of Elliptic Curve Cryptography algorithm that can test if following X9.62 technology standard specification. The validation tool can be applied all information securities using DES, SEED, AES, SHA-1/256/384/512, RSA-OAEP V2.0, V2.1, ECDSA, ECKCDSA, ECDH, etc. Moreover, we can enhance the precision of validation through several experiments and perform the validation tool in the online environment.

A Strategy for Validation in Preliminary Design Stage using The Simulation of Model Behavior (모델 행동 양식의 시뮬레이션을 이용한 초기 디자인 검증 방법)

  • Shin, Seung-Hun;Park, Seung-Kyu;Choi, Kyung-Hee
    • Journal of the Korea Society for Simulation
    • /
    • v.19 no.1
    • /
    • pp.153-160
    • /
    • 2010
  • Most part of errors in software development process are included during the stage of requirements definition and design. And correction or elimination of errors from those stages requires much more efforts and costs than those from the later part of software development process. However, despite of the importance of the validation of requirement definition and design stages, several kinds of problem have made it hard to be done successfully. Therefore, in this paper, we introduce a novel validation process for the preliminary design stage. The validation process is based on simulations of model and it can be used to validate requirements and model simultaneously. Models in the validation process will take only the behavior of software and be built on Ptolemy framework. The usability of our validation process is confirmed with a case study over DNS system environment. And the result of simulation shows well-known errors or vulnerabilities can be found with simulations of model which has the behavior of software. This means our validation process can be used as a process to validate requirements and models during the early stage of software development process.

Validation of the Revised Class Play Scale for the Assessment of the Quality of Peer Relations (또래관계의 질 평가를 위한 RCP(Revised Class Play)의 타당화 연구)

  • Chun, Hui Young
    • Korean Journal of Child Studies
    • /
    • v.20 no.2
    • /
    • pp.3-19
    • /
    • 1999
  • The Revised Classroom Play(RCP) scale was validated for Korean 4th grade students. 104 boys and 80 girls took the translated RCP. Test-retest reliability was established with 41 children after 2 months, and test of criterion-related validation was established by the Social Behavior Scale administered to 71 children. All raw scores were standardized by classroom and sex to adjust for differences in the number of nominators and nominatees. The structure of the RCP resulted in 3 factors, 'sociability-disruptive' and 'shy-isolated', with 28 items similar to the original RCP 3 factors. For criterion-related validity, the correlations between the 3 factors of the RCP and the 3 factors of the Social Behavior Scale were positive. For internal consistencies, Cronbach's ${\alpha}$ of the RCP factors ranged from .77 to .94. Test-retest reliability coefficients ranged from .33 to .51. Analyses of variance revealed good item discrepancies for all 28 items. These results confirm the RCP as a valid and reliable measure of the quality of peer relations.

  • PDF

Experimental validation of ASME strain-based seismic assessment methods using piping elbow test data

  • Jong-Min Lee ;Jae-Yoon Kim;Hyun-Seok Song ;Yun-Jae Kim ;Jin-Weon Kim
    • Nuclear Engineering and Technology
    • /
    • v.55 no.5
    • /
    • pp.1616-1629
    • /
    • 2023
  • To quantify the conservatism of existing ASME strain-based evaluation methods for seismic loading, this paper presents very low cycle fatigue test data of elbows under various cyclic loading conditions and comparison of evaluation results with experimental failure cycles. For strain-based evaluation methods, the method presented in ASME BPVC CC N-900 and Sec. VIII are used. Predicted failure cycles are compared with experimental failure cycle to quantify the conservatism of evaluation methods. All methods give very conservative failure cycles. The CC N-900 method is the most conservative and prediction results are only ~0.5% of experimental data. For Sec. VIII method, the use of the option using code tensile properties gives ~3% of experimental data, and the use of the material-specific reduction of area can reduce conservatism but still gives ~15% of experimental data.

Establishment of Validation Methods to Test the Biocompatibility of Titanium Dioxide

  • Kim, Mi-Ju;Lim, Hee-Joung;Lee, Byung Gun;Kim, Jong-Hoon;Choi, Jinsub;Kang, Hee-Gyoo
    • Bulletin of the Korean Chemical Society
    • /
    • v.34 no.6
    • /
    • pp.1857-1863
    • /
    • 2013
  • Most of biomaterials come in direct contact with the body, making standardized methods of evaluation and validation of biocompatibility an important aspect to biomaterial development. However, biomaterial validation guidelines have not been fully established, until now. This study was to compare the in vitro behavior of osteoblasts cultured on nanomaterial $TiO_2$ surfaces to osteoblast behavior on culture plates. Comparisons were also made to cells grown in conditioned media (CM) that creates an environment similar to the in vivo environment. Comparisons were made between the different growth conditions for osteoblast adhesion, proliferation, differentiation, and functionality. We found that the in vivo-like system of growing cells in concentrated CM provided a good validation method for biomaterial development and in vivo implant therapy. The $TiO_2$ materials were biocompatible, showing similar behavior to that observed in vivo. This study provided valuable information that would aid in the creation of guidelines into standardization and evaluation of biocompatibility in $TiO_2$ biomaterials.

Cross-cultural Validation of Instruments Measuring Health Beliefs about Colorectal Cancer Screening among Korean Americans

  • Lee, Shin-Young;Lee, Eunice E.
    • Journal of Korean Academy of Nursing
    • /
    • v.45 no.1
    • /
    • pp.129-138
    • /
    • 2015
  • Purpose: The purpose of this study was to report the instrument modification and validation processes to make existing health belief model scales culturally appropriate for Korean Americans (KAs) regarding colorectal cancer (CRC) screening utilization. Methods: Instrument translation, individual interviews using cognitive interviewing, and expert reviews were conducted during the instrument modification phase, and a pilot test and a cross-sectional survey were conducted during the instrument validation phase. Data analyses of the cross-sectional survey included internal consistency and construct validity using exploratory and confirmatory factor analysis. Results: The main issues identified during the instrument modification phase were (a) cultural and linguistic translation issues and (b) newly developed items reflecting Korean cultural barriers. Cross-sectional survey analyses during the instrument validation phase revealed that all scales demonstrate good internal consistency reliability (Cronbach's alpha=.72~.88). Exploratory factor analysis showed that susceptibility and severity loaded on the same factor, which may indicate a threat variable. Items with low factor loadings in the confirmatory factor analysis may relate to (a) lack of knowledge about fecal occult blood testing and (b) multiple dimensions of the subscales. Conclusion: Methodological, sequential processes of instrument modification and validation, including translation, individual interviews, expert reviews, pilot testing and a cross-sectional survey, were provided in this study. The findings indicate that existing instruments need to be examined for CRC screening research involving KAs.

Development of the Algorithm for Optimizing Wavelength Selection in Multiple Linear Regression

  • Hoeil Chung
    • Near Infrared Analysis
    • /
    • v.1 no.1
    • /
    • pp.1-7
    • /
    • 2000
  • A convenient algorithm for optimizing wavelength selection in multiple linear regression (MLR) has been developed. MOP (MLP Optimization Program) has been developed to test all possible MLR calibration models in a given spectral range and finally find an optimal MLR model with external validation capability. MOP generates all calibration models from all possible combinations of wavelength, and simultaneously calculates SEC (Standard Error of Calibration) and SEV (Standard Error of Validation) by predicting samples in a validation data set. Finally, with determined SEC and SEV, it calculates another parameter called SAD (Sum of SEC, SEV, and Absolute Difference between SEC and SEV: sum(SEC+SEV+Abs(SEC-SEV)). SAD is an useful parameter to find an optimal calibration model without over-fitting by simultaneously evaluating SEC, SEV, and difference of error between calibration and validation. The calibration model corresponding to the smallest SAD value is chosen as an optimum because the errors in both calibration and validation are minimal as well as similar in scale. To evaluate the capability of MOP, the determination of benzene content in unleaded gasoline has been examined. MOP successfully found the optimal calibration model and showed the better calibration and independent prediction performance compared to conventional MLR calibration.

Automatic RF Input Power Level Control Methodology for SAR Measurement Validation

  • Kim, Ki-Hwea;Choi, Dong-Geun;Gimm, Yoon-Myoung
    • Journal of electromagnetic engineering and science
    • /
    • v.15 no.3
    • /
    • pp.181-184
    • /
    • 2015
  • Evaluation of radiating radiofrequency fields from hand-held and body-mounted wireless communication devices to human bodies are conducted by measuring the specific absorption rate (SAR). The uncertainty of system validation and probe calibration in SAR measurement depend on the variation of RF power used for the validation and calibration. RF input power for system validation or probe calibration is controlled manually during the test process of the existing systems in the laboratories. Consequently, a long time is required to reach the stable power needed for testing that will cause less uncertainty. The standard uncertainty due to this power drift is typically 2.89%, which can be obtained by applying IEC 62209 in a normal operating condition. The principle of the Automatic Input Power Level Control System (AIPLC), which controls the equipment by a program that maintains a stable input power level, is suggested in this paper. The power drift is reduced to less than ${\pm}1.16dB$ by AIPLC, which reduces the standard uncertainty of power drift to 0.67%.