• Korean Journal of Radiology
• /
• v.22 no.3
• /
• pp.442-453
• /
• 2021

Artificial intelligence (AI) will likely affect various fields of medicine. This article aims to explain the fundamental principles of clinical validation, device approval, and insurance coverage decisions of AI algorithms for medical diagnosis and prediction. Discrimination accuracy of AI algorithms is often evaluated with the Dice similarity coefficient, sensitivity, specificity, and traditional or free-response receiver operating characteristic curves. Calibration accuracy should also be assessed, especially for algorithms that provide probabilities to users. As current AI algorithms have limited generalizability to real-world practice, clinical validation of AI should put it to proper external testing and assisting roles. External testing could adopt diagnostic case-control or diagnostic cohort designs. A diagnostic case-control study evaluates the technical validity/accuracy of AI while the latter tests the clinical validity/accuracy of AI in samples representing target patients in real-world clinical scenarios. Ultimate clinical validation of AI requires evaluations of its impact on patient outcomes, referred to as clinical utility, and for which randomized clinical trials are ideal. Device approval of AI is typically granted with proof of technical validity/accuracy and thus does not intend to directly indicate if AI is beneficial for patient care or if it improves patient outcomes. Neither can it categorically address the issue of limited generalizability of AI. After achieving device approval, it is up to medical professionals to determine if the approved AI algorithms are beneficial for real-world patient care. Insurance coverage decisions generally require a demonstration of clinical utility that the use of AI has improved patient outcomes.

• Journal of Korean Society on Water Environment
• /
• v.29 no.3
• /
• pp.365-376
• /
• 2013

The purpose of this study was to evaluate effects of spatio-temporal changes in land uses and rainfall magnitude using the Soil and Water Assessment Tool (SWAT). Prior of application of the model to real-world problem, the model should be calibrated and validated properly. In most modeling approaches, the validation process is done assuming no significant changes occurring at the study watershed between calibration and validation periods, which is not proper assumption for agricultural watersheds. If simulated results obtained with calibrated parameters match observed data with higher accuracy for validation period, this does not always mean the simulated result represents rainfall-runoff, pollutant generation and transport mechanism for validation period because temporal and spatial variables and rainfall magnitude are often not the same. In this study SWAT was applied to Mandae study watershed in Korea to evaluate effects of spatio-temporal changes in landuses using 2009 and 2010 crop data for each field at the watershed. The Nash-Sutcliffe model efficiency (NSE) values for calibration and validation with either 2009 or 2010 was evaluated and the NSE value for calibration with 2009 and calibration with 2010 were compared. It was found that if there is substantial change in land use and rainfall, model calibration period should be determined to reflect those changes. Through these approaches, inherent limitation of the SWAT, which does not consider changes in land uses over the simulation period, was investigated. Also, Effects of changes in rainfall magnitude during calibration process were analyzed.

• Korean Journal of Remote Sensing
• /
• v.39 no.5_1
• /
• pp.577-587
• /
• 2023

The utilization of crowdsourced spatial data has been actively researched; however, issues stemming from the uncertainty of data quality have been raised. In particular, when low-quality data is mixed into drone imagery datasets, it can degrade the quality of spatial information output. In order to address these problems, the study presents a methodology for automatically validating the geometric quality of crowdsourced imagery. Key quality factors such as spatial resolution, resolution variation, matching point reprojection error, and bundle adjustment results are utilized. To classify imagery suitable for spatial information generation, training and validation datasets are constructed, and machine learning is conducted using a radial basis function (RBF)-based support vector machine (SVM) model. The trained SVM model achieved a classification accuracy of 99.1%. To evaluate the effectiveness of the quality validation model, imagery sets before and after applying the model to drone imagery not used in training and validation are compared by generating orthoimages. The results confirm that the application of the quality validation model reduces various distortions that can be included in orthoimages and enhances object identifiability. The proposed quality validation methodology is expected to increase the utility of crowdsourced data in spatial information generation by automatically selecting high-quality data from the multitude of crowdsourced data with varying qualities.

• Communications for Statistical Applications and Methods
• /
• v.15 no.2
• /
• pp.283-292
• /
• 2008

Smoothing parameter or bandwidth plays a key role in nonparametric classification based on kernel density estimation. We consider choosing smoothing parameter in nonparametric classification, which optimize the Bayes risk. Hall and Kang (2005) clarified the theoretical properties of smoothing parameter in terms of minimizing Bayes risk and derived the optimal order of it. Bootstrap method was used in their exploring numerical properties. We compare cross-validation and bootstrap method numerically in terms of optimal order of bandwidth. Effects on misclassification rate are also examined. We confirm that bootstrap method is superior to cross-validation in both cases.

• Journal of Soil and Groundwater Environment
• /
• v.15 no.5
• /
• pp.8-15
• /
• 2010

The purpose of this research is to check the validation of an auto-analysis method combined with aqua regia digestion apparatus for improvement of water distillation method used as a fluorine test of soil. Fluorine contents of CaO used in the pretreatment course of water distillation method were 120 mg/kg ~ 5,064 mg/kg at the blank test, which was exceeded up to maximum 12.5 times of the soil standard, so it was estimated due to a effect of fluorine existing as impurities of CaO. The recovery test of the same samples indicated that water distillation method and auto-analysis method were 134.5mg/kg and 161.7mg/kg respectively, the recovery ratio of the latter was 16.8% higher than the formal. The validation test of two methods satisfied the standard, but auto analysis method was excellent more than distillation method. Also, auto analysis method could save a analysis time up to maximum 4.7 times by comparison with water distillation method.

• Environmental Analysis Health and Toxicology
• /
• v.20 no.4 s.51
• /
• pp.297-302
• /
• 2005

The purpose of this study was to provide the standard method for the analysis of organophosphorous pesticides such as chlorpyrifos, diazinon, malathion and parathion in blood. We performed method validation for these pesticides in blood according to EURACHEM (A focus For Analytical Chemistry in Europe) guide. For the analysis of the pesticides, we used solid-phase extraction ,column (Waters Oasis $HLB^{(R)}$. After the extraction, the supernatants were evaporated to dryness under the nitrogen stream. They were analyzed by gas chromatography/mass spectrometry (GC/MS) after reconstituting with ethanol. Terbufos was used as an internal standard. To validate this method, we performed verification procedures with the following parameters: selectivity, linearity of calibration, accuracy, precision, limit of detection and quantification. Validation data according to Eurachem guide were adequate for our purpose for the analysis of chlorpyrifos, diazinon, malathion and parathion in blood.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2007.06a
• /
• pp.653-656
• /
• 2007

A concern about the ebXML concentrated, the necessity for Validation request among the ebXML sollutions has become both at home and abroad. There aren't any test implement for Validation of BPSS. The ebXML BPSS system establishment related to ebMS Standard Conformance and Interoperability is required. In this paper, We'll suggest system model to estimate Conformance of document in accordance with case items to frame the ebXML BPSS Validation items.

• Journal of Mechanical Science and Technology
• /
• v.17 no.12
• /
• pp.1844-1854
• /
• 2003

This paper discusses details of modeling and robust control of an AMB (active magnetic bearing) spindle, and part I presents a modeling and validation process of the AMB spindle. There are many components in AMB spindle : electromagnetic actuator, sensor, rotor, power amplifier and digital controller. If each component is carefully modeled and evaluated, the components have tight structured uncertainty bounds and achievable performance of the system increases. However, since some unknown dynamics may exist and the augmented plant could show some discrepancy with the real plant, the validation of the augmented plant is needed through measuring overall frequency responses of the actual plant. In addition, it is necessary to combine several components and identify them with a reduced order model. First, all components of the AMB spindle are carefully modeled and identified based on experimental data, which also render valuable information in quantifying structured uncertainties. Since sensors, power amplifiers and discretization dynamics can be considered as time delay components, such dynamics are combined and identified with a reduced order. Then, frequency responses of the open-loop plant are measured through closed-loop experiments to validate the augmented plant. The whole modeling process gives an accurate nominal model of a low order for the robust control design.

• Proceedings of the Korean Association of Geographic Inforamtion Studies Conference
• /
• 2003.04a
• /
• pp.291-296
• /
• 2003

This study estimated surface temperature by using split-window technique and NOAA/AVHRR data was used. For surface monitoring, cloud masking procedure was carried out using threshold algorithm. The daily maximum air temperature is estimated by multiple regression method using independent variables such as satellite-derived surface temperature, EDD, and latitude. When the EDD data added, the highest correlation shown. This indicates that EDD data is the necessary element for estimation of the daily maximum air temperature. We derived correlation and experience equation by three approaching method to estimate daily maximum air temperature. 1) non-considering landcover method as season, 2) considering landcover method as season, and 3) just method as landcover. The last approaching method shows the highest correlation. So cross-validation procedure was used in third method for validation of the estimated value. For all landcover type 5, the results using the cross-validation procedure show reasonable agreement with measured values(slope=0.97, intercept=-0.30, R$^2$=0.84, RMSE=4.24$^{\circ}C$). Also, for all landcover type 7, the results using the cross-validation procedure show reasonable agreement with measured values(slope=0.993, Intercept=0.062, R$^2$=0.84, RMSE=4.43$^{\circ}C$).

• International Journal of Railway
• /
• v.2 no.3
• /
• pp.99-106
• /
• 2009

Checks and tests before putting safety facilities into service as well as the results of these tests are essential, time consuming and may show great variations between each other. Economic constraints and the increasing complexity associated with the development of computerized tools tend to limit the capacity of the classic approval process (manual or automatic). A reduction of the validation cover rate could result in practice. This is not compatible with the French national plan to renew the interlocking systems of the national network. The method and the tool presented in this paper makes it possible to formally validate new computerized systems or evolutions of existing French interlocking systems with real-time functional interpreted Petri nets. The aim of our project is to provide SNCF with a method for the formal validation of French interlocking systems. A formal proof method by assertion, which is applicable to industrial automation equipment such as interlocking systems, and which covers equally the specification and its real software implementation, is presented in this paper. With the proposed method we completely verify that the system follows all safety properties at all times and does not show superfluous conditions: it replaces all the indoor checks (not the outdoor checks). The advantages expected are a significant reduction of testing time and of the related costs, an increase of the test coverage rate, an answer to the new demand of railway infrastructure maintenance engineering to modify and validate computerized interlocking systems. Formal methods mastery by infrastructure engineers are surely a key to prove that more safety is not necessarily more expensive.