• 제목/요약/키워드: metrics validation

검색결과 64건 처리시간 0.022초

RISKY MODULE PREDICTION FOR NUCLEAR I&C SOFTWARE

  • Kim, Young-Mi;Kim, Hyeon-Soo
    • Nuclear Engineering and Technology
    • /
    • 제44권6호
    • /
    • pp.663-672
    • /
    • 2012
  • As software based digital I&C (Instrumentation and Control) systems are used more prevalently in nuclear plants, enhancement of software dependability has become an important issue in the area of nuclear I&C systems. Critical attributes of software dependability are safety and reliability. These attributes are tightly related to software failures caused by faults. Software testing and V&V (Verification and Validation) activities are hence important for enhancing software dependability. If the risky modules of safety-critical software can be predicted, it will be possible to focus on testing and V&V activities more efficiently and effectively. It should also make it possible to better allocate resources for regulation activities. We propose a prediction technique to estimate risky software modules by adopting machine learning models based on software complexity metrics. An empirical study with various machine learning algorithms was executed for comparing the prediction performance. Experimental results show SVMs (Support Vector Machines) perform as well or better than the other methods.

소프트웨어 품질측정을 위한 소프트웨어 품질매트릭 방법론과 적용 연구 ((A Study on Software Quality Metric Methodology and Application for Software Quality Measurement))

  • 이성기
    • 한국국방경영분석학회지
    • /
    • 제22권2호
    • /
    • pp.90-112
    • /
    • 1996
  • Research issues in software engineering in recent may be object oriented methodology and software quality. Since Halstead has proposed metric-software science in 1977, software quality area has been studied in steady but inactively until 1980s. As international standards such as ISO 9000-3, 9126 were enacted in 1990s early, interest in software quality is increased but many problems such as how to validate metric, measure quality or apply metric are remained. This paper proposes software quality metric methodology which software developer or project manager can use in measuring quality and validating metric during software development. The methodology is classified by several phases: establishment of quality requirement, identification of quality metric, data collection, metric implementation, metric validation. In order to show its applicability, test program, metrics and data are applied to each phase of the methodology. Consideration of this methodology as a methodology for software quality measurement similar to development methodology for software development is needed.

  • PDF

Safe Discharge Criteria After Curative Gastrectomy for Gastric Cancer

  • Guner, Ali;Kim, Ki Yoon;Park, Sung Hyun;Cho, Minah;Kim, Yoo Min;Hyung, Woo Jin;Kim, Hyoung-Il
    • Journal of Gastric Cancer
    • /
    • 제22권4호
    • /
    • pp.395-407
    • /
    • 2022
  • Purpose: This study aimed to investigate the relationship between clinical and laboratory parameters and complication status to predict which patients can be safely discharged from the hospital on the third postoperative day (POD). Materials and Methods: Data from a prospectively maintained database of 2,110 consecutive patients with gastric adenocarcinoma who underwent curative surgery were reviewed. The third POD vital signs, laboratory data, and details of the course after surgery were collected. Patients with grade II or higher complications after the third POD were considered unsuitable for early discharge. The performance metrics were calculated for all algorithm parameters. The proposed algorithm was tested using a validation dataset of consecutive patients from the same center. Results: Of 1,438 patients in the study cohort, 142 (9.9%) were considered unsuitable for early discharge. C-reactive protein level, body temperature, pulse rate, and neutrophil count had good performance metrics and were determined to be independent prognostic factors. An algorithm consisting of these 4 parameters had a negative predictive value (NPV) of 95.9% (95% confidence interval [CI], 94.2-97.3), sensitivity of 80.3% (95% CI, 72.8-86.5), and specificity of 51.1% (95% CI, 48.3-53.8). Only 28 (1.9%) patients in the study cohort were classified as false negatives. In the validation dataset, the NPV was 93.7%, sensitivity was 66%, and 3.3% (17/512) of patients were classified as false negatives. Conclusions: Simple clinical and laboratory parameters obtained on the third POD can be used when making decisions regarding the safe early discharge of patients who underwent gastrectomy.

객체지향 메트릭을 이용한 결함 예측 모형의 실험적 비교 (A Comparative Experiment of Software Defect Prediction Models using Object Oriented Metrics)

  • 김윤규;김태연;채흥석
    • 한국정보과학회논문지:컴퓨팅의 실제 및 레터
    • /
    • 제15권8호
    • /
    • pp.596-600
    • /
    • 2009
  • 검증과 확인을 통한 소프트웨어의 효율적인 관리를 지원하기 위하여 객체지향 메트릭 기반의 결함 예측 모형이 많이 제안되고 있다. 제안된 모형은 주로 로지스틱 회귀분석으로 개발하였다. 그리고 개발된 모형의 결함 예측 정확도는 60${\sim}$70%이었다. 본 논문에서는 기존 결함 예측 모형의 효과를 확인하기 위하여 이클립스 3.3을 대상으로 개발된 모형과 유사한 방법으로 실험을 하였다. 실험 결과 모형의 정확성은 약 40%이었다. 이는 주장된 예측력보다 많이 낮은 수치이었다. 또한 단순 로지스틱 회귀분석이 다중 로지스틱 회귀분석보다 높은 예측력을 보였다.

체험형 교육 서비스 품질 측정 항목에 관한 연구: 창의적 체험활동을 중심으로 (EEPERF(Experiential Education PERFormance): An Instrument for Measuring Service Quality in Experiential Education)

  • 박기윤;김현식
    • 유통과학연구
    • /
    • 제10권2호
    • /
    • pp.43-52
    • /
    • 2012
  • 최근 국내외에서 체험형 교육서비스의 도입과 활용이 활발해지고 있다. 체험형 교육서비스가 제대로 관리됨으로써 효과적인 체험형 교육서비스가 제공될 경우 다양한 체험형 교육서비스 제공자와 이용자 사이의 사회적, 경제적 교환의 기회가 지속적으로 확대될 개연성이 존재한다. 이를 위해 필요한 선결과제 중 하나가 바로 '성과 측정 수단'인데, 문제는 새로운 체험형 교육서비스 성과 측정에 적합한 성과 측정 수단이 마땅치 않다는 점이다. 이와 같은 상황에서 우선적으로 고려해볼 수 있는 연구화두는 다음과 같다: 창의적 체험활동과 같은 체험형 교육 서비스 품질을 어떻게 측정할 것인가. 본 연구에서는 바로 이 화두에 대한 답을 모색하기 위해 체험형 교육 서비스 중 하나인 창의적 체험활동 교육 서비스 품질 측정 문항을 개발하여 체험형 교육서비스 품질 측정 도구화 하는 것을 목적으로 연구를 진행하였다. 본 연구에서는 체험형 교육 서비스 품질 평가를 위한 척도를 개발하기 위해 창의적 체험활동에 초점을 맞추어 이론적 배경으로부터 실무 전문가의 검토 조정을 거쳐 평가 척도 후보군을 개발하고, 실증적 정제 과정을 통해 최종 척도 항목을 도출하였다. 본 연구에서 도출한 체험형 교육 서비스 품질 측정 항목은 (체험형 교육) 결과 품질 (EE-outcome), (체험형 교육) 공감 품질 (EE-empathy), (체험형 교육) 신뢰 품질 (EE-reliability), (체험형 교육) 물리적환경 품질 (EE-scape) 등 4차원으로 구성되며, 전반적인 신뢰성과 타당성이 있는 것으로 확인되었다. 본 연구에서 제시하는 체험형 교육 서비스 품질 측정 도구는 운용 주체에게 기획 포인트를 제공하고 '교육용' 세분시장을 추구하는 서비스 제공 주체에게는 관리 포인트를 제공한다는 점에서 유용한 지침이 될 수 있을 것이다.

  • PDF

객체지향 메트릭을 이용한 결함 예측 모형의 임계치 설정에 관한 실험 (An Experiment for Determining Threshold of Defect Prediction Models using Object Oriented Metrics)

  • 김윤규;채흥석
    • 한국정보과학회논문지:컴퓨팅의 실제 및 레터
    • /
    • 제15권12호
    • /
    • pp.943-947
    • /
    • 2009
  • 소프트웨어의 결함을 예측하고 검증과 확인 활동을 통하여 효율적인 자원을 관리하기 위하여 많은 연구에서 결함 예측 모형을 제안하고 있다. 하지만 기존의 연구는 예측율이 최대 효과를 보이는 임계치에 결함 예측 모형의 예측율을 평가하고 있다. 이는 측정 시스템의 결함 정보를 알고 있는 가정하에서 평가가 이루어지는 것이기 때문에 실제 결함 정보를 알 수 없는 시스템에서는 최적의 임계치를 결정할 수 없다. 그러므로 임계치 선정의 중요성을 확인하기 위하여 본 연구에서는 결함 예측 모형으로 타 시스템의 결함을 예측하는 비교 실험을 하였다. 실험은 기존에 제안된 3개의 결함 예측 모형과 4개의 시스템을 대상으로 하였고 결함 예측 모형의 임계치별 예측의 정확성을 비교하였다. 실험결과에서 임계치는 모형의 예측율과 높은 관련이 있었지만 실제 결함 정보가 확인 안 되는 시스템에 대하여 결함을 예측하는 경우에는 임계치를 선정할 수 없음을 확인하였다. 따라서 결함 예측 모형을 타 시스템에 적용하기 위하석 임계치 선정에 관한 추후 연구가 필요함을 확인하였다.

텍스트 분류 기반 기계학습의 정신과 진단 예측 적용 (Application of Text-Classification Based Machine Learning in Predicting Psychiatric Diagnosis)

  • 백두현;황민규;이민지;우성일;한상우;이연정;황재욱
    • 생물정신의학
    • /
    • 제27권1호
    • /
    • pp.18-26
    • /
    • 2020
  • Objectives The aim was to find effective vectorization and classification models to predict a psychiatric diagnosis from text-based medical records. Methods Electronic medical records (n = 494) of present illness were collected retrospectively in inpatient admission notes with three diagnoses of major depressive disorder, type 1 bipolar disorder, and schizophrenia. Data were split into 400 training data and 94 independent validation data. Data were vectorized by two different models such as term frequency-inverse document frequency (TF-IDF) and Doc2vec. Machine learning models for classification including stochastic gradient descent, logistic regression, support vector classification, and deep learning (DL) were applied to predict three psychiatric diagnoses. Five-fold cross-validation was used to find an effective model. Metrics such as accuracy, precision, recall, and F1-score were measured for comparison between the models. Results Five-fold cross-validation in training data showed DL model with Doc2vec was the most effective model to predict the diagnosis (accuracy = 0.87, F1-score = 0.87). However, these metrics have been reduced in independent test data set with final working DL models (accuracy = 0.79, F1-score = 0.79), while the model of logistic regression and support vector machine with Doc2vec showed slightly better performance (accuracy = 0.80, F1-score = 0.80) than the DL models with Doc2vec and others with TF-IDF. Conclusions The current results suggest that the vectorization may have more impact on the performance of classification than the machine learning model. However, data set had a number of limitations including small sample size, imbalance among the category, and its generalizability. With this regard, the need for research with multi-sites and large samples is suggested to improve the machine learning models.

Feature Selection with Ensemble Learning for Prostate Cancer Prediction from Gene Expression

  • Abass, Yusuf Aleshinloye;Adeshina, Steve A.
    • International Journal of Computer Science & Network Security
    • /
    • 제21권12spc호
    • /
    • pp.526-538
    • /
    • 2021
  • Machine and deep learning-based models are emerging techniques that are being used to address prediction problems in biomedical data analysis. DNA sequence prediction is a critical problem that has attracted a great deal of attention in the biomedical domain. Machine and deep learning-based models have been shown to provide more accurate results when compared to conventional regression-based models. The prediction of the gene sequence that leads to cancerous diseases, such as prostate cancer, is crucial. Identifying the most important features in a gene sequence is a challenging task. Extracting the components of the gene sequence that can provide an insight into the types of mutation in the gene is of great importance as it will lead to effective drug design and the promotion of the new concept of personalised medicine. In this work, we extracted the exons in the prostate gene sequences that were used in the experiment. We built a Deep Neural Network (DNN) and Bi-directional Long-Short Term Memory (Bi-LSTM) model using a k-mer encoding for the DNA sequence and one-hot encoding for the class label. The models were evaluated using different classification metrics. Our experimental results show that DNN model prediction offers a training accuracy of 99 percent and validation accuracy of 96 percent. The bi-LSTM model also has a training accuracy of 95 percent and validation accuracy of 91 percent.

Validation of a Cognitive Task Simulation and Rehearsal Tool for Open Carpal Tunnel Release

  • Paro, John A.M.;Luan, Anna;Lee, Gordon K.
    • Archives of Plastic Surgery
    • /
    • 제44권3호
    • /
    • pp.223-227
    • /
    • 2017
  • Background Carpal tunnel release is one of the most common surgical procedures performed by hand surgeons. The authors created a surgical simulation of open carpal tunnel release utilizing a mobile and rehearsal platform app. This study was performed in order to validate the simulator as an effective training platform for carpal tunnel release. Methods The simulator was evaluated using a number of metrics: construct validity (the ability to identify variability in skill levels), face validity (the perceived ability of the simulator to teach the intended material), content validity (that the simulator was an accurate representation of the intended operation), and acceptability validity (willingness of the desired user group to adopt this method of training). Novices and experts were recruited. Each group was tested, and all participants were assigned an objective score, which served as construct validation. A Likert-scale questionnaire was administered to gauge face, content, and acceptability validity. Results Twenty novices and 10 experts were recruited for this study. The objective performance scores from the expert group were significantly higher than those of the novice group, with surgeons scoring a median of 74% and medical students scoring a median of 45%. The questionnaire responses indicated face, content, and acceptability validation. Conclusions This mobile-based surgical simulation platform provides step-by-step instruction for a variety of surgical procedures. The findings of this study help to demonstrate its utility as a learning tool, as we confirmed construct, face, content, and acceptability validity for carpal tunnel release. This easy-to-use educational tool may help bring surgical education to a new- and highly mobile-level.

Novel nomogram-based integrated gonadotropin therapy individualization in in vitro fertilization/intracytoplasmic sperm injection: A modeling approach

  • Ebid, Abdel Hameed IM;Motaleb, Sara M Abdel;Mostafa, Mahmoud I;Soliman, Mahmoud MA
    • Clinical and Experimental Reproductive Medicine
    • /
    • 제48권2호
    • /
    • pp.163-173
    • /
    • 2021
  • Objective: This study aimed to characterize a validated model for predicting oocyte retrieval in controlled ovarian stimulation (COS) and to construct model-based nomograms for assistance in clinical decision-making regarding the gonadotropin protocol and dose. Methods: This observational, retrospective, cohort study included 636 women with primary unexplained infertility and a normal menstrual cycle who were attempting assisted reproductive therapy for the first time. The enrolled women were split into an index group (n=497) for model building and a validation group (n=139). The primary outcome was absolute oocyte count. The dose-response relationship was tested using modified Poisson, negative binomial, hybrid Poisson-Emax, and linear models. The validation group was similarly analyzed, and its results were compared to that of the index group. Results: The Poisson model with the log-link function demonstrated superior predictive performance and precision (Akaike information criterion, 2,704; λ=8.27; relative standard error (λ)=2.02%). The covariate analysis included women's age (p<0.001), antral follicle count (p<0.001), basal follicle-stimulating hormone level (p<0.001), gonadotropin dose (p=0.042), and protocol type (p=0.002 and p<0.001 for short and antagonist protocols, respectively). The estimates from 500 bootstrap samples were close to those of the original model. The validation group showed model assessment metrics comparable to the index model. Based on the fitted model, a static nomogram was built to improve visualization. In addition, a dynamic electronic tool was created for convenience of use. Conclusion: Based on our validated model, nomograms were constructed to help clinicians individualize the stimulation protocol and gonadotropin doses in COS cycles.