• 제목/요약/키워드: External Validation

검색결과 205건 처리시간 0.022초

Predictive Modeling for the Growth of Salmonella Enterica Serovar Typhimurium on Lettuce Washed with Combined Chlorine and Ultrasound During Storage

  • Park, Shin Young;Zhang, Cheng Yi;Ha, Sang-Do
    • 한국식품위생안전성학회지
    • /
    • 제34권4호
    • /
    • pp.374-379
    • /
    • 2019
  • 본 연구에서는 대표적인 신선 잎채소류인 상추의 세척 단계에서 초음파 (37 kHz) 와 염소 (100~300 ppm) 의 병용처리 후 냉장 ~ 실온저장 ($10{\sim}25^{\circ}C$)에 따른 이 식품 중의 Salmonella Typhimurium의 성장예측모델을 개발하였다. 1 차 모델 개발을 위해 Gompertz 방정식을 활용하여 각기 다른 실험 조건에서의 S. Typhimurium의 생육도 (SGR 과 LT)를 조사했다. 본 방정식에 의한 1 차 모델 개발시 $R^2$가 0.92 이상으로 우수하게 나타났으며 저장온도가 낮을수록 초음파에 사용된 염소의 농도가 높을수록 SGR 값은 감소하였고 LT 값은 증가하였다. 이를 바탕으로 2 차 polynomial 모델을 개발하여 다양한 통계적 지표 ($R^2$, MSE, $A_f$$B_f$)를 통해 분석한 결과 개발된 모델의 적합성을 확인할 수 있었다. 따라서 개발된 모델이 초음파와 염소의 병용 세척에 따른 저장 중 상추에 대한 S. Typhimurium의 성장예측모델로 사용 가능하다고 판단되어지며, 신선 잎채소류에서의 식중독을 예방하고 미생물학적 위생관리기준을 설정하는데 기초자료로 활용될 수 있을 것으로 사료된다.

교차검증을 이용한 국내선 항공수요예측 (Domestic air demand forecast using cross-validation)

  • 임재환;김영록;최연철;김광일
    • 한국항공운항학회지
    • /
    • 제27권1호
    • /
    • pp.43-50
    • /
    • 2019
  • The aviation demand forecast field has been actively studied along with the recent growth of the aviation market. In this study, the demand for domestic passenger demand and freight demand was estimated through cross-validation method. As a result, passenger demand is influenced by private consumption growth rate, oil price, and exchange rate. Freight demand is affected by GDP per capita, private consumption growth rate, and oil price. In particular, passenger demand is characterized by temporary external shocks, and freight demand is more affected by economic variables than temporary shocks.

Development and Validation of a Model Using Radiomics Features from an Apparent Diffusion Coefficient Map to Diagnose Local Tumor Recurrence in Patients Treated for Head and Neck Squamous Cell Carcinoma

  • Minjae Kim;Jeong Hyun Lee;Leehi Joo;Boryeong Jeong;Seonok Kim;Sungwon Ham;Jihye Yun;NamKug Kim;Sae Rom Chung;Young Jun Choi;Jung Hwan Baek;Ji Ye Lee;Ji-hoon Kim
    • Korean Journal of Radiology
    • /
    • 제23권11호
    • /
    • pp.1078-1088
    • /
    • 2022
  • Objective: To develop and validate a model using radiomics features from apparent diffusion coefficient (ADC) map to diagnose local tumor recurrence in head and neck squamous cell carcinoma (HNSCC). Materials and Methods: This retrospective study included 285 patients (mean age ± standard deviation, 62 ± 12 years; 220 male, 77.2%), including 215 for training (n = 161) and internal validation (n = 54) and 70 others for external validation, with newly developed contrast-enhancing lesions at the primary cancer site on the surveillance MRI following definitive treatment of HNSCC between January 2014 and October 2019. Of the 215 and 70 patients, 127 and 34, respectively, had local tumor recurrence. Radiomics models using radiomics scores were created separately for T2-weighted imaging (T2WI), contrast-enhanced T1-weighted imaging (CE-T1WI), and ADC maps using non-zero coefficients from the least absolute shrinkage and selection operator in the training set. Receiver operating characteristic (ROC) analysis was used to evaluate the diagnostic performance of each radiomics score and known clinical parameter (age, sex, and clinical stage) in the internal and external validation sets. Results: Five radiomics features from T2WI, six from CE-T1WI, and nine from ADC maps were selected and used to develop the respective radiomics models. The area under ROC curve (AUROC) of ADC radiomics score was 0.76 (95% confidence interval [CI], 0.62-0.89) and 0.77 (95% CI, 0.65-0.88) in the internal and external validation sets, respectively. These were significantly higher than the AUROC values of T2WI (0.53 [95% CI, 0.40-0.67], p = 0.006), CE-T1WI (0.53 [95% CI, 0.40-0.67], p = 0.012), and clinical parameters (0.53 [95% CI, 0.39-0.67], p = 0.021) in the external validation set. Conclusion: The radiomics model using ADC maps exhibited higher diagnostic performance than those of the radiomics models using T2WI or CE-T1WI and clinical parameters in the diagnosis of local tumor recurrence in HNSCC following definitive treatment.

예측모형의 구축과 검증: 소화기암연구 사례를 중심으로 (Development and Validation of a Prediction Model: Application to Digestive Cancer Research)

  • 권용한;한경화
    • Journal of Digestive Cancer Research
    • /
    • 제11권3호
    • /
    • pp.157-164
    • /
    • 2023
  • Prediction is a significant topic in clinical research. The development and validation of a prediction model has been increasingly published in clinical research. In this review, we investigated analytical methods and validation schemes for a clinical prediction model used in digestive cancer research. Deep learning and logistic regression, with split-sample validation as an internal or external validation, were the most commonly used methods. Furthermore, we briefly introduced and summarized the advantages and disadvantages of each method. Finally, we discussed several points to consider when conducting prediction model studies.

Key Principles of Clinical Validation, Device Approval, and Insurance Coverage Decisions of Artificial Intelligence

  • Seong Ho Park;Jaesoon Choi;Jeong-Sik Byeon
    • Korean Journal of Radiology
    • /
    • 제22권3호
    • /
    • pp.442-453
    • /
    • 2021
  • Artificial intelligence (AI) will likely affect various fields of medicine. This article aims to explain the fundamental principles of clinical validation, device approval, and insurance coverage decisions of AI algorithms for medical diagnosis and prediction. Discrimination accuracy of AI algorithms is often evaluated with the Dice similarity coefficient, sensitivity, specificity, and traditional or free-response receiver operating characteristic curves. Calibration accuracy should also be assessed, especially for algorithms that provide probabilities to users. As current AI algorithms have limited generalizability to real-world practice, clinical validation of AI should put it to proper external testing and assisting roles. External testing could adopt diagnostic case-control or diagnostic cohort designs. A diagnostic case-control study evaluates the technical validity/accuracy of AI while the latter tests the clinical validity/accuracy of AI in samples representing target patients in real-world clinical scenarios. Ultimate clinical validation of AI requires evaluations of its impact on patient outcomes, referred to as clinical utility, and for which randomized clinical trials are ideal. Device approval of AI is typically granted with proof of technical validity/accuracy and thus does not intend to directly indicate if AI is beneficial for patient care or if it improves patient outcomes. Neither can it categorically address the issue of limited generalizability of AI. After achieving device approval, it is up to medical professionals to determine if the approved AI algorithms are beneficial for real-world patient care. Insurance coverage decisions generally require a demonstration of clinical utility that the use of AI has improved patient outcomes.

Risk Prediction Using Genome-Wide Association Studies on Type 2 Diabetes

  • Choi, Sungkyoung;Bae, Sunghwan;Park, Taesung
    • Genomics & Informatics
    • /
    • 제14권4호
    • /
    • pp.138-148
    • /
    • 2016
  • The success of genome-wide association studies (GWASs) has enabled us to improve risk assessment and provide novel genetic variants for diagnosis, prevention, and treatment. However, most variants discovered by GWASs have been reported to have very small effect sizes on complex human diseases, which has been a big hurdle in building risk prediction models. Recently, many statistical approaches based on penalized regression have been developed to solve the "large p and small n" problem. In this report, we evaluated the performance of several statistical methods for predicting a binary trait: stepwise logistic regression (SLR), least absolute shrinkage and selection operator (LASSO), and Elastic-Net (EN). We first built a prediction model by combining variable selection and prediction methods for type 2 diabetes using Affymetrix Genome-Wide Human SNP Array 5.0 from the Korean Association Resource project. We assessed the risk prediction performance using area under the receiver operating characteristic curve (AUC) for the internal and external validation datasets. In the internal validation, SLR-LASSO and SLR-EN tended to yield more accurate predictions than other combinations. During the external validation, the SLR-SLR and SLR-EN combinations achieved the highest AUC of 0.726. We propose these combinations as a potentially powerful risk prediction model for type 2 diabetes.

Quality Control of Pharmacopuncture: A Comparative Study of Good Manufacturing Practice and External Herbal Dispensary Standards

  • Han, Ji-Eun;Park, Minjung;An, Tteul-E-Bom;Park, Jong-Hyun;Oh, Danny;Kim, Kyeong Han;Sung, Soo-Hyun
    • 대한약침학회지
    • /
    • 제24권2호
    • /
    • pp.59-67
    • /
    • 2021
  • Objectives: We aimed to compare the external herbal dispensary (EHD) evaluation criteria for pharmacopuncture and the Korea Good Manufacturing Practice (KGMP) sterile medicine standards to contribute to the establishment of quality control criteria for pharmacopuncture. Methods: We obtained the KGMP standards from the Ministry of Food and Drug Safety and the pharmacopuncture certification criteria from the Ministry of Health and Welfare of South Korea. The EHD evaluation items were classified into three categories: facilities, quality control, and validation. The evaluation items were compared with the KGMP sterile medicine criteria to determine their conformance with each other, followed by a discussion among the committee of six experts and their consensus to suggest the items to complement the EHD evaluation criteria. Results: Among the KGMP sterile medicine criteria, 44 were related to the management of the facilities, and 32 pharmacopuncture evaluation items corresponded to these KGMP items (66.7%). Fifty-eight KGMP criteria were related to quality management, and 42 pharmacopuncture evaluation items corresponded to these KGMP items (72.4%). Twentyfive KGMP sterile medicine criteria were related to validation, and 11 pharmacopuncture evaluation items corresponded to these KGMP items (44.0%). Sixteen items under the pharmacopuncture EHD criteria corresponded to the KGMP sterile medicine criteria based on the consent of the experts. Among these, 4 were related to facility management, 6 were related to quality control, and 6 were related to validation. Conclusion: For the safety and quality control of pharmacopuncture, there is a need to select the criteria for the mandatory items among the proposed pharmacopuncture-EHD criteria laws and systems to ensure that the pharmacopuncture materials are produced under the pharmacopuncture-EHD in compliance with the relevant requirements. More studies are needed to secure the safety level of pharmacopuncture materials corresponding to that of conventional medicine.

Chest Radiography of Tuberculosis: Determination of Activity Using Deep Learning Algorithm

  • Ye Ra Choi;Soon Ho Yoon;Jihang Kim;Jin Young Yoo;Hwiyoung Kim;Kwang Nam Jin
    • Tuberculosis and Respiratory Diseases
    • /
    • 제86권3호
    • /
    • pp.226-233
    • /
    • 2023
  • Background: Inactive or old, healed tuberculosis (TB) on chest radiograph (CR) is often found in high TB incidence countries, and to avoid unnecessary evaluation and medication, differentiation from active TB is important. This study develops a deep learning (DL) model to estimate activity in a single chest radiographic analysis. Methods: A total of 3,824 active TB CRs from 511 individuals and 2,277 inactive TB CRs from 558 individuals were retrospectively collected. A pretrained convolutional neural network was fine-tuned to classify active and inactive TB. The model was pretrained with 8,964 pneumonia and 8,525 normal cases from the National Institute of Health (NIH) dataset. During the pretraining phase, the DL model learns the following tasks: pneumonia vs. normal, pneumonia vs. active TB, and active TB vs. normal. The performance of the DL model was validated using three external datasets. Receiver operating characteristic analyses were performed to evaluate the diagnostic performance to determine active TB by DL model and radiologists. Sensitivities and specificities for determining active TB were evaluated for both the DL model and radiologists. Results: The performance of the DL model showed area under the curve (AUC) values of 0.980 in internal validation, and 0.815 and 0.887 in external validation. The AUC values for the DL model, thoracic radiologist, and general radiologist, evaluated using one of the external validation datasets, were 0.815, 0.871, and 0.811, respectively. Conclusion: This DL-based algorithm showed potential as an effective diagnostic tool to identify TB activity, and could be useful for the follow-up of patients with inactive TB in high TB burden countries.

Development and Validation of a Deep Learning System for Segmentation of Abdominal Muscle and Fat on Computed Tomography

  • Hyo Jung Park;Yongbin Shin;Jisuk Park;Hyosang Kim;In Seob Lee;Dong-Woo Seo;Jimi Huh;Tae Young Lee;TaeYong Park;Jeongjin Lee;Kyung Won Kim
    • Korean Journal of Radiology
    • /
    • 제21권1호
    • /
    • pp.88-100
    • /
    • 2020
  • Objective: We aimed to develop and validate a deep learning system for fully automated segmentation of abdominal muscle and fat areas on computed tomography (CT) images. Materials and Methods: A fully convolutional network-based segmentation system was developed using a training dataset of 883 CT scans from 467 subjects. Axial CT images obtained at the inferior endplate level of the 3rd lumbar vertebra were used for the analysis. Manually drawn segmentation maps of the skeletal muscle, visceral fat, and subcutaneous fat were created to serve as ground truth data. The performance of the fully convolutional network-based segmentation system was evaluated using the Dice similarity coefficient and cross-sectional area error, for both a separate internal validation dataset (426 CT scans from 308 subjects) and an external validation dataset (171 CT scans from 171 subjects from two outside hospitals). Results: The mean Dice similarity coefficients for muscle, subcutaneous fat, and visceral fat were high for both the internal (0.96, 0.97, and 0.97, respectively) and external (0.97, 0.97, and 0.97, respectively) validation datasets, while the mean cross-sectional area errors for muscle, subcutaneous fat, and visceral fat were low for both internal (2.1%, 3.8%, and 1.8%, respectively) and external (2.7%, 4.6%, and 2.3%, respectively) validation datasets. Conclusion: The fully convolutional network-based segmentation system exhibited high performance and accuracy in the automatic segmentation of abdominal muscle and fat on CT images.