• Title/Summary/Keyword: Accuracy Statistics

Search Result 823, Processing Time 0.026 seconds

Comparative Analysis of Anomaly Detection Models using AE and Suggestion of Criteria for Determining Outliers

  • Kang, Gun-Ha;Sohn, Jung-Mo;Sim, Gun-Wu
    • Journal of the Korea Society of Computer and Information
    • /
    • v.26 no.8
    • /
    • pp.23-30
    • /
    • 2021
  • In this study, we present a comparative analysis of major autoencoder(AE)-based anomaly detection methods for quality determination in the manufacturing process and a new anomaly discrimination criterion. Due to the characteristics of manufacturing site, anomalous instances are few and their types greatly vary. These properties degrade the performance of an AI-based anomaly detection model using the dataset for both normal and anomalous cases, and incur a lot of time and costs in obtaining additional data for performance improvement. To solve this problem, the studies on AE-based models such as AE and VAE are underway, which perform anomaly detection using only normal data. In this work, based on Convolutional AE, VAE, and Dilated VAE models, statistics on residual images, MSE, and information entropy were selected as outlier discriminant criteria to compare and analyze the performance of each model. In particular, the range value applied to the Convolutional AE model showed the best performance with AUC PRC 0.9570, F1 Score 0.8812 and AUC ROC 0.9548, accuracy 87.60%. This shows a performance improvement of an accuracy about 20%P(Percentage Point) compared to MSE, which was frequently used as a standard for determining outliers, and confirmed that model performance can be improved according to the criteria for determining outliers.

Improving the Classification of Population and Housing Census with AI: An Industry and Job Code Study

  • Byung-Il Yun;Dahye Kim;Young-Jin Kim;Medard Edmund Mswahili;Young-Seob Jeong
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.4
    • /
    • pp.21-29
    • /
    • 2023
  • In this paper, we propose an AI-based system for automatically classifying industry and occupation codes in the population census. The accurate classification of industry and occupation codes is crucial for informing policy decisions, allocating resources, and conducting research. However, this task has traditionally been performed by human coders, which is time-consuming, resource-intensive, and prone to errors. Our system represents a significant improvement over the existing rule-based system used by the statistics agency, which relies on user-entered data for code classification. In this paper, we trained and evaluated several models, and developed an ensemble model that achieved an 86.76% match accuracy in industry and 81.84% in occupation, outperforming the best individual model. Additionally, we propose process improvement work based on the classification probability results of the model. Our proposed method utilizes an ensemble model that combines transfer learning techniques with pre-trained models. In this paper, we demonstrate the potential for AI-based systems to improve the accuracy and efficiency of population census data classification. By automating this process with AI, we can achieve more accurate and consistent results while reducing the workload on agency staff.

Feasibility of Single-Shot Whole Thoracic Time-Resolved MR Angiography to Evaluate Patients with Multiple Pulmonary Arteriovenous Malformations

  • Jihoon Hong;Sang Yub Lee;Jae-Kwang Lim;Jongmin Lee;Jongmin Park;Jung Guen Cha;Hui Joong Lee;Donghyeon Kim
    • Korean Journal of Radiology
    • /
    • v.23 no.8
    • /
    • pp.794-802
    • /
    • 2022
  • Objective: To evaluate the feasibility of single-shot whole thoracic time-resolved MR angiography (TR-MRA) to identify the feeding arteries of pulmonary arteriovenous malformations (PAVMs) and reperfusion of the lesion after embolization in patients with multiple PAVMs. Materials and Methods: Nine patients (8 females and 1 male; age range, 23-65 years) with a total of 62 PAVMs who underwent percutaneous embolization for multiple PAVMs and were subsequently followed up using TR-MRA and CT obtained within 6 months from each other were retrospectively reviewed. All imaging analyses were performed by two independent readers blinded to clinical information. The visibility of the feeding arteries on maximum intensity projection (MIP) reconstruction and multiplanar reconstruction (MPR) TR-MRA images was evaluated by comparing them to CT as a reference. The accuracy of TR-MRA for diagnosing reperfusion of the PAVM after embolization was assessed in a subgroup with angiographic confirmation. The reliability between the readers in interpreting the TR-MRA results was analyzed using kappa (κ) statistics. Results: Feeding arteries were visible on the original MIP images of TR-MRA in 82.3% (51/62) and 85.5% (53/62) of readers 1 and 2, respectively. Using the MPR, the rates increased to 93.5% (58/62) and 95.2% (59/62), respectively (κ = 0.760 and 0.792, respectively). Factors for invisibility were the course of feeding arteries in the anteroposterior plane, proximity to large enhancing vessels, adjacency to the chest wall, pulsation of the heart, and small feeding arteries. Thirty-seven PAVMs in five patients had angiographic confirmation of reperfusion status after embolization (32 occlusions and 5 reperfusions). TR-MRA showed 100% (5/5) sensitivity and 100% (32/32, including three cases in which the feeding arteries were not visible on TR-MRA) specificity for both readers. Conclusion: Single-shot whole thoracic TR-MRA with MPR showed good visibility of the feeding arteries of PAVMs and high accuracy in diagnosing reperfusion after embolization. Single-shot whole thoracic TR-MRA may be a feasible method for the follow-up of patients with multiple PAVMs.

SonazoidTM versus SonoVue® for Diagnosing Hepatocellular Carcinoma Using Contrast-Enhanced Ultrasound in At-Risk Individuals: A Prospective, Single-Center, Intraindividual, Noninferiority Study

  • Hyo-Jin Kang;Jeong Min Lee;Jeong Hee Yoon;Jeongin Yoo;Yunhee Choi;Ijin Joo;Joon Koo Han
    • Korean Journal of Radiology
    • /
    • v.23 no.11
    • /
    • pp.1067-1077
    • /
    • 2022
  • Objective: To determine whether Sonazoid-enhanced ultrasound (SZUS) was noninferior to SonoVue-enhanced ultrasound (SVUS) in diagnosing hepatocellular carcinoma (HCC) using the same diagnostic criteria. Materials and Methods: This prospective, single-center, noninferiority study (NCT04847726) enrolled 105 at-risk participants (71 male; mean age ± standard deviation, 63 ± 11 years; range, 26-86 years) with treatment-naïve solid hepatic nodules (≥ 1 cm). All participants underwent same-day SZUS (experimental method) and SVUS (control method) for one representative nodule per participant. Images were interpreted by three readers (the operator and two independent readers). All malignancies were diagnosed histopathologically, while the benignity of other lesions was confirmed by follow-up stability or pathology. The primary endpoint was per-lesion diagnostic accuracy for HCC pooled across three readers using the conventional contrast-enhanced ultrasound diagnostic criteria, including arterial phase hyperenhancement followed by mild (assessed within 2 minutes after contrast injection) and late (≥ 60 seconds with a delay of 5 minutes) washout. The noninferiority delta was -10%p. Furthermore, different time delays were compared as washout criteria in SZUS, including delays of 2, 5, and > 10 minutes. Results: A total of 105 lesions (HCCs [n = 61], non-HCC malignancies [n = 19], and benign [n = 25]) were evaluated. Using the 5-minutes washout criterion, per-lesion accuracy of SZUS pooled across the three readers (72.4%; 95% confidence interval [CI], 64.1%-79.3%) was noninferior to that of SVUS (71.4%; 95% CI, 63.1%-78.6%), meeting the statistical criterion for non-inferiority (difference of 0.95%p; 95% CI, -3.8%p-5.7%p). The arterial phase hyperenhancement combined with the 5-minutes washout criterion showed the same sensitivity as that of the > 10-minutes criterion (59.0% vs. 59.0%, p = 0.989), and the specificities were not significantly different (90.9% vs. 86.4%, p = 0.072). Conclusion: SZUS was noninferior to SVUS for diagnosing HCC in at-risk patients using the same diagnostic criteria. No significant improvement in HCC diagnosis was observed by extending the washout time delay from 5 to 10 minutes.

Diagnostic Performance of Cardiac CT and Transthoracic Echocardiography for Detection of Surgically Confirmed Bicuspid Aortic Valve: Effect of Calcium Extent and Valve Subtypes (외과적으로 확진된 이첨 대동맥 판막의 진단을 위한 심장 CT 및 경흉부 심초음파의 진단적 성능: 판막 아형 및 칼슘의 양이 미치는 효과)

  • Jeongju Kim;Sung Mok Kim;Joonghyun Ahn;Jihoon Kim;Yeon Hyeon Choe
    • Journal of the Korean Society of Radiology
    • /
    • v.84 no.6
    • /
    • pp.1324-1336
    • /
    • 2023
  • Purpose This study aimed to compare the diagnostic performance of cardiac CT and transthoracic echocardiogram (TTE) depending on the degree of valvular calcification and bicuspid aortic valve (BAV) subtype. Materials and Methods This retrospective study included 266 consecutive patients (106 with BAV and 160 with tricuspid aortic valve) who underwent cardiac CT and TTE before aortic valve replacement. Cardiac CT was used to evaluate the morphology of the aortic valve, and a calcium scoring scan was used to quantify valve calcium. The aortic valves were classified into fused and two-sinus types. The diagnostic accuracy of cardiac CT and TTE was calculated using a reference standard for intraoperative inspection. Results CT demonstrated significantly higher sensitivity, negative predictive value, and accuracy than TTE in detecting BAV (p < 0.001, p < 0.001, and p = 0.003, respectively). The TTE sensitivity tended to decrease as valvular calcification increased. The error rate of TTE for CT was 10.9% for the twosinus type of BAV and 28.3% for the fused type (p = 0.044). Conclusion Cardiac CT had a higher diagnostic performance in detecting BAV than TTE and may help diagnose BAV, particularly in patients with severe valvular calcification.

Development of an AI Model to Determine the Relationship between Cerebrovascular Disease and the Work Environment as well as Analysis of Consistency with Expert Judgment (뇌심혈관 질환과 업무 환경의 연관성 판단을 위한 AI 모델의 개발 및 전문가 판단과의 일치도 분석)

  • Juyeon Oh;Ki-bong Yoo;Ick Hoon Jin;Byungyoon Yun;Juho Sim;Heejoo Park;Jongmin Lee;Jian Lee;Jin-Ha Yoon
    • Journal of Korean Society of Occupational and Environmental Hygiene
    • /
    • v.34 no.3
    • /
    • pp.202-213
    • /
    • 2024
  • Introduction: Acknowledging the global issue of diseases potentially caused by overwork, this study aims to develop an AI model to help workers understand the connection between cerebrocardiovascular diseases and their work environment. Materials and methods: The model was trained using medical and legal expertise along with data from the 2021 occupational disease adjudication certificate by the Industrial Accident Compensation Insurance and Prevention Service. The Polyglot-ko-5.8B model, which is effective for processing Korean, was utilized. Model performance was evaluated through accuracy, precision, sensitivity, and F1-score metrics. Results: The model trained on a comprehensive dataset, including expert knowledge and actual case data, outperformed the others with respective accuracy, precision, sensitivity, and F1-scores of 0.91, 0.89, 0.84, and 0.87. However, it still had limitations in responding to certain scenarios. Discussion: The comprehensive model proved most effective in diagnosing work-related cerebrocardiovascular diseases, highlighting the significance of integrating actual case data in AI model development. Despite its efficacy, the model showed limitations in handling diverse cases and offering health management solutions. Conclusion: The study succeeded in creating an AI model to discern the link between work factors and cerebrocardiovascular diseases, showcasing the highest efficacy with the comprehensively trained model. Future enhancements towards a template-based approach and the development of a user-friendly chatbot webUI for workers are recommended to address the model's current limitations.

Preliminary Feasibility Study for Korean Lung Capacity Prediction Formula: Focused on Statistical Test Model (한국인 폐활량 예측산식을 위한 예비타당성 연구: 통계검정모델 중심)

  • Myungmo Lee;Younjung Oh;Samho Park;Weechang Kang
    • Journal of Korean Physical Therapy Science
    • /
    • v.31 no.3
    • /
    • pp.31-50
    • /
    • 2024
  • Background: The lung capacity prediction formula in Korea is an important judgment standard. Since there is no appropriate lung capacity prediction formula, various prediction formulas are used for foreigners such as Northeast Asians. The purpose of this study is to develop a lung capacity prediction equation by selecting data and setting the selection criteria for normal subjects in accordance with international standards through strict quality control, and to propose a new prediction model. Design: Preliminary feasibility study Methods: A total of 857 people who met the criteria for normal people were finally collected. The tester used for the lung capacity test was the V-Max Encore 22 (Carefusion, California, USA), which is a lung capacity tester proposed by the Korean Society of Tuberculosis and Respiratory Medicine and satisfies accuracy and precision. Among the indicators measured using spirometry, forced vital capacity (FVC), forced expiratory volume in 1 second (FEV1), forced expiratory volume ratio in 1 second (FEV1/FVC), forced mid-expiratory flow (Forced expiratory flow 25-75%, FEF25-75%) and peak expiratory flow (PEF) values were collected. Results: This study confirmed a significant correlation between age, height, weight, and pulmonary function indicators. Additionally, it found a correlation between body mass index, which considers the diversity of physical conditions, and pulmonary function indicators. Graphs depicting age-specific pulmonary function indicators by gender, presented as generalized additive model results from collected data, showed a pattern where both FVC and FEV1 increased until the mid-20s and then gradually decreased with aging. FEV1% and PEF exhibited a continuous decrease with aging. Conclusion: This study confirms that there is a significant correlation between weight and pulmonary function in the prediction formula for lung capacity. Additionally, it verifies the correlation between body mass index, which considers the diversity of physical conditions, and pulmonary function. The study suggests that the predicted values are relatively low due to factors such as aging and environmental influences like COVID-19. This preliminary study holds clinical significance for improving the diagnostic accuracy of respiratory symptoms in the elderly.

Ensemble Learning with Support Vector Machines for Bond Rating (회사채 신용등급 예측을 위한 SVM 앙상블학습)

  • Kim, Myoung-Jong
    • Journal of Intelligence and Information Systems
    • /
    • v.18 no.2
    • /
    • pp.29-45
    • /
    • 2012
  • Bond rating is regarded as an important event for measuring financial risk of companies and for determining the investment returns of investors. As a result, it has been a popular research topic for researchers to predict companies' credit ratings by applying statistical and machine learning techniques. The statistical techniques, including multiple regression, multiple discriminant analysis (MDA), logistic models (LOGIT), and probit analysis, have been traditionally used in bond rating. However, one major drawback is that it should be based on strict assumptions. Such strict assumptions include linearity, normality, independence among predictor variables and pre-existing functional forms relating the criterion variablesand the predictor variables. Those strict assumptions of traditional statistics have limited their application to the real world. Machine learning techniques also used in bond rating prediction models include decision trees (DT), neural networks (NN), and Support Vector Machine (SVM). Especially, SVM is recognized as a new and promising classification and regression analysis method. SVM learns a separating hyperplane that can maximize the margin between two categories. SVM is simple enough to be analyzed mathematical, and leads to high performance in practical applications. SVM implements the structuralrisk minimization principle and searches to minimize an upper bound of the generalization error. In addition, the solution of SVM may be a global optimum and thus, overfitting is unlikely to occur with SVM. In addition, SVM does not require too many data sample for training since it builds prediction models by only using some representative sample near the boundaries called support vectors. A number of experimental researches have indicated that SVM has been successfully applied in a variety of pattern recognition fields. However, there are three major drawbacks that can be potential causes for degrading SVM's performance. First, SVM is originally proposed for solving binary-class classification problems. Methods for combining SVMs for multi-class classification such as One-Against-One, One-Against-All have been proposed, but they do not improve the performance in multi-class classification problem as much as SVM for binary-class classification. Second, approximation algorithms (e.g. decomposition methods, sequential minimal optimization algorithm) could be used for effective multi-class computation to reduce computation time, but it could deteriorate classification performance. Third, the difficulty in multi-class prediction problems is in data imbalance problem that can occur when the number of instances in one class greatly outnumbers the number of instances in the other class. Such data sets often cause a default classifier to be built due to skewed boundary and thus the reduction in the classification accuracy of such a classifier. SVM ensemble learning is one of machine learning methods to cope with the above drawbacks. Ensemble learning is a method for improving the performance of classification and prediction algorithms. AdaBoost is one of the widely used ensemble learning techniques. It constructs a composite classifier by sequentially training classifiers while increasing weight on the misclassified observations through iterations. The observations that are incorrectly predicted by previous classifiers are chosen more often than examples that are correctly predicted. Thus Boosting attempts to produce new classifiers that are better able to predict examples for which the current ensemble's performance is poor. In this way, it can reinforce the training of the misclassified observations of the minority class. This paper proposes a multiclass Geometric Mean-based Boosting (MGM-Boost) to resolve multiclass prediction problem. Since MGM-Boost introduces the notion of geometric mean into AdaBoost, it can perform learning process considering the geometric mean-based accuracy and errors of multiclass. This study applies MGM-Boost to the real-world bond rating case for Korean companies to examine the feasibility of MGM-Boost. 10-fold cross validations for threetimes with different random seeds are performed in order to ensure that the comparison among three different classifiers does not happen by chance. For each of 10-fold cross validation, the entire data set is first partitioned into tenequal-sized sets, and then each set is in turn used as the test set while the classifier trains on the other nine sets. That is, cross-validated folds have been tested independently of each algorithm. Through these steps, we have obtained the results for classifiers on each of the 30 experiments. In the comparison of arithmetic mean-based prediction accuracy between individual classifiers, MGM-Boost (52.95%) shows higher prediction accuracy than both AdaBoost (51.69%) and SVM (49.47%). MGM-Boost (28.12%) also shows the higher prediction accuracy than AdaBoost (24.65%) and SVM (15.42%)in terms of geometric mean-based prediction accuracy. T-test is used to examine whether the performance of each classifiers for 30 folds is significantly different. The results indicate that performance of MGM-Boost is significantly different from AdaBoost and SVM classifiers at 1% level. These results mean that MGM-Boost can provide robust and stable solutions to multi-classproblems such as bond rating.

Characterizing the Thermal Stability of TiSi2 Film by Using the Statistical Experimental Method (통계적 실험 방법을 이용한 티타늄실리사이드의 열적안정성 연구)

  • Cheong, Seong-Hwee;Song, Oh-Sung
    • Korean Journal of Materials Research
    • /
    • v.13 no.3
    • /
    • pp.200-204
    • /
    • 2003
  • A statistical experiment method was employed to investigate the window of the thermal stability of $TiSi_2$films which are popular for Ti-salicide and ohmic layers. The statistical experimental results showed that the first order term of $TiSi_2$thickness and annealing temperature was acceptable as a function of $\Delta$resistivity by 95% reliability criteria, and R-sq value implying a fit accuracy of the model also showed a high value of 93.80%. We found that $\Delta$resistivity of the $TiSi_2$film annealed at $700^{\circ}C$ for 1 hr changed from 3.35 to $0.379\mu$$\Omega$$\cdot$cm with increasing thickness from 185 to $703\AA$, and TEX>$\Delta$resistivity of the $TiSi_2$film with a fixed thickness of 444 $\AA$ changed from 0.074 to 17.12 $\mu$$\Omega$$\cdot$cm with increasing temperature increase from 600 to $800^{\circ}C$. From these results, we report that the process conditions of$ 692^{\circ}C$-1 hr, $715^{\circ}C$-1 hr, and 73$0^{\circ}C$-1 hr for $TiSi_2$($400 \AA$) are stable by the criteria of 1, 2, and 3 $\mu$$\Omega$$\cdot$cm of $\Delta$resistivity, respectively.

LEONHARD EULER (1707-1783) AND THE COMPUTATIONAL ASPECTS OF SOME ZETA-FUNCTION SERIES

  • Srivastava, Hari Mohan
    • Journal of the Korean Mathematical Society
    • /
    • v.44 no.5
    • /
    • pp.1163-1184
    • /
    • 2007
  • In this presentation dedicated to the tricentennial birth anniversary of the great eighteenth-century Swiss mathematician, Leonhard Euler (1707-1783), we begin by remarking about the so-called Basler problem of evaluating the Zeta function ${\zeta}(s)$ [in the much later notation of Georg Friedrich Bernhard Riemann (1826-1866)] when s=2, which was then of vital importance to Euler and to many other contemporary mathematicians including especially the Bernoulli brothers [Jakob Bernoulli (1654-1705) and Johann Bernoulli (1667-1748)], and for which a fascinatingly large number of seemingly independent solutions have appeared in the mathematical literature ever since Euler first solved this problem in the year 1736. We then investigate various recent developments on the evaluations and representations of ${\zeta}(s)$ when $s{\in}{\mathbb{N}}{\backslash}\;[1],\;{\mathbb{N}}$ being the set of natural numbers. We emphasize upon several interesting classes of rapidly convergent series representations for ${\zeta}(2n+1)(n{\in}{\mathbb{N}})$ which have been developed in recent years. In two of many computationally useful special cases considered here, it is observed that ${\zeta}(3)$ can be represented by means of series which converge much more rapidly than that in Euler's celebrated formula as well as the series used recently by Roger $Ap\'{e}ry$ (1916-1994) in his proof of the irrationality of ${\zeta}(3)$. Symbolic and numerical computations using Mathematica (Version 4.0) for Linux show, among other things, that only 50 terms of one of these series are capable of producing an accuracy of seven decimal places.