• 제목/요약/키워드: Data Quality Validation

검색결과 379건 처리시간 0.034초

2차원 관로형 지하시설물 정보 품질검증기술 개발 (Development of 2D Data Quality Validation Techniques for Pipe-type Underground Facilities)

  • 배상근;김상민;유은진;임거배;정다운
    • 산업경영시스템학회지
    • /
    • 제46권3호
    • /
    • pp.285-292
    • /
    • 2023
  • As various accidents have occurred in underground spaces, we aim to improve the quality validation standards and methods as specified in the Regulations on Producing Integrated Map of Underground Spaces devised by the Ministry of Land, Infrastructure and Transport of the Republic of Korea for a high-quality integrated map of underground spaces. Specifically, we propose measures to improve the quality assurance of pipeline-type underground facilities, the so-called life lines given their importance for citizens' daily activities and their highest risk of accident among the 16 types of underground facilities. After implementing quality validation software based on the developed quality validation standards, the adequacy of the validation standards was demonstrated by testing using data from two-dimensional water supply facilities in some areas of Busan, Korea. This paper has great significance in that it has laid the foundation for reducing the time and manpower required for data quality inspection and improving data quality reliability by improving current quality validation standards and developing technologies that can automatically extract errors through software.

한국의 기온자료 품질관리 알고리즘의 검증 (Validation of Quality Control Algorithms for Temperature Data of the Republic of Korea)

  • 박창용;최영은
    • 대기
    • /
    • 제22권3호
    • /
    • pp.299-307
    • /
    • 2012
  • This study is aimed to validate errors for detected suspicious temperature data using various quality control procedures for 61 weather stations in the Republic of Korea. The quality control algorithms for temperature data consist of four main procedures (high-low extreme check, internal consistency check, temporal outlier check, and spatial outlier check). Errors of detected suspicious temperature data are judged by examining temperature data of nearby stations, surface weather charts, hourly temperature data, daily precipitation, and daily maximum wind direction. The number of detected errors in internal consistency check and spatial outlier check showed 4 days (3 stations) and 7 days (5 stations), respectively. Effective and objective methods for validation errors through this study will help to reduce manpower and time for conduct of quality management for temperature data.

인공지능 데이터 품질검증 기술 및 오픈소스 프레임워크 분석 연구 (An Evaluation Study on Artificial Intelligence Data Validation Methods and Open-source Frameworks)

  • 윤창희;신호경;추승연;김재일
    • 한국멀티미디어학회논문지
    • /
    • 제24권10호
    • /
    • pp.1403-1413
    • /
    • 2021
  • In this paper, we investigate automated data validation techniques for artificial intelligence training, and also disclose open-source frameworks, such as Google's TensorFlow Data Validation (TFDV), that support automated data validation in the AI model development process. We also introduce an experimental study using public data sets to demonstrate the effectiveness of the open-source data validation framework. In particular, we presents experimental results of the data validation functions for schema testing and discuss the limitations of the current open-source frameworks for semantic data. Last, we introduce the latest studies for the semantic data validation using machine learning techniques.

BIM 모델의 품질검증 사례연구 (Case Study of BIM Quality Assurance)

  • 정연석;이상일;이상호
    • 한국전산구조공학회:학술대회논문집
    • /
    • 한국전산구조공학회 2010년도 정기 학술대회
    • /
    • pp.379-382
    • /
    • 2010
  • This study proposes a way to validate BIM data quality in BIM applications. Solibri model checker is adopted as a module development platform, which is based on Java programming language. The platform makes application developers implement BIM model checker for their own purpose. This study has developed a BIM validation module for circulation analysis of building design. The validation module enables end-users to automatically detect data corrupted or not defined. In case studies, the module found that an IFC file generated from a BIM software has wrong relation information between a space and boundary elements. A building model should satisfy modeling requirements and then domain users can get analysis results. The BIM data validation module needs to be developed in each BIM application domain.

  • PDF

크라우드소싱 드론 영상의 기하학적 품질 자동 검증 (Automatic Validation of the Geometric Quality of Crowdsourcing Drone Imagery)

  • 이동호;최경아
    • 대한원격탐사학회지
    • /
    • 제39권5_1호
    • /
    • pp.577-587
    • /
    • 2023
  • 크라우드소싱(crowdsourcing) 공간 데이터 활용 연구가 활발히 진행되고 있으나 데이터 품질의 불확실성으로 인한 문제점이 제기되고 있다. 특히 드론 영상 데이터셋에 품질이 낮은 데이터가 포함될 경우, 출력되는 공간 정보의 품질이 저하될 수 있다. 이를 위해 본 연구에서는 크라우드소싱된 영상의 기하학적 품질을 자동으로 검증하는 방법론을 제안하였다. 주요 품질 요소로는 영상의 공간해상도, 해상도 변화량, 매칭점 재투영 오차, 번들 조정 결과 등을 입력변수로 활용하였다. 공간 정보 생성에 적합한 영상을 분류하기 위해 학습 및 검증 데이터를 구축하고, radial basis function (RBF) 기반의 support vector machine (SVM) 모델로 학습을 진행하였다. 학습된 SVM 모델의 분류 정확도는 99.1%를 기록하였다. 품질 검증 모델 효과를 확인하기 위해 학습 및 검증에 사용하지 않은 드론 영상에 대하여 해당 모델을 적용하기 전후의 영상 데이터셋으로 각각 정사영상을 생성하고 비교하였다. 그 결과 모델 적용을 통하여 정사영상에 포함될 수 있는 다양한 왜곡을 줄이고 객체 식별력을 증대시키는 것을 확인하였다. 제안된 품질 검증 방법론은 다양한 품질의 크라우드소싱 데이터를 입력으로 받아 양질의 정보만을 자동 선별하게 함으로써 공간정보 생성에서의 활용 가능성을 증대시킬 것으로 기대한다.

HVAC 파라미터 모니터링 시스템에 대한 고찰 (Computer Validation 중심으로) (A Study on HVAC Parameter Monitoring System (Regarding Computer Validation))

  • 김종구
    • 대한설비공학회:학술대회논문집
    • /
    • 대한설비공학회 2008년도 하계학술발표대회 논문집
    • /
    • pp.90-95
    • /
    • 2008
  • This article presents practical advice regarding the implementation and management of an impeccable Building Management System. The BMS was introduced to the series of computerized systems including manufacturing, storage, distribution, and quality control. Recently revised GMP regulation is requesting an improvement in drug product quality regulatory system by computer system validation. Quality is critical to guarantee the efficacy and the safety of drugs and is approved in the evaluation process after the audit trail application. HVAC parameter monitoring system will record the identity of operators entering or confirming critical data. Authority to amend entered data should be restricted to nominated persons. Any alteration to an entry of critical data should be authorized in advance and recorded with the reason for the change.

  • PDF

Finding Unexpected Test Accuracy by Cross Validation in Machine Learning

  • Yoon, Hoijin
    • International Journal of Computer Science & Network Security
    • /
    • 제21권12spc호
    • /
    • pp.549-555
    • /
    • 2021
  • Machine Learning(ML) splits data into 3 parts, which are usually 60% for training, 20% for validation, and 20% for testing. It just splits quantitatively instead of selecting each set of data by a criterion, which is very important concept for the adequacy of test data. ML measures a model's accuracy by applying a set of validation data, and revises the model until the validation accuracy reaches on a certain level. After the validation process, the complete model is tested with the set of test data, which are not seen by the model yet. If the set of test data covers the model's attributes well, the test accuracy will be close to the validation accuracy of the model. To make sure that ML's set of test data works adequately, we design an experiment and see if the test accuracy of model is always close to its validation adequacy as expected. The experiment builds 100 different SVM models for each of six data sets published in UCI ML repository. From the test accuracy and its validation accuracy of 600 cases, we find some unexpected cases, where the test accuracy is very different from its validation accuracy. Consequently, it is not always true that ML's set of test data is adequate to assure a model's quality.

A Study on Quality Checking of National Scholar Content DB

  • Kim, Byung-Kyu;Choi, Seon-Hee;Kim, Jay-Hoon;You, Beom-Jong
    • International Journal of Contents
    • /
    • 제6권3호
    • /
    • pp.1-4
    • /
    • 2010
  • The national management and retrieval service of the national scholar Content DB are very important. High quality content can improve the user's utilization and satisfaction and be a strong base for both the citation index creation and the calculation of journal impact factors. Therefore, the system is necessary to check data quality effectively. We have closely studied and developed a webbased data quality checking system that will support anything from raw digital data to its automatic validation as well as hands-on validation, all of which will be discussed in this paper.

QbD6시그마 프로세스를 통한 D-항원 정량 시험법의 유효성과 동등성에 관한 연구 (A Study on the Efficacy and Equivalence of D-antigen Quantitative Analysis through QbD6sigma Process)

  • 김강희;김현정
    • 품질경영학회지
    • /
    • 제50권4호
    • /
    • pp.831-842
    • /
    • 2022
  • Purpose: This study carried out the Quality by Design (QbD)6σ process to verify the effectiveness and equivalence of the finished D-antigen quantitative test method, and compared the OFAT-based method validation and test result acceptance criteria with the Analytical Quality by Design (AQbD)-based method validation and test method. This is a study on how to reduce the risk of delay in permit change by increasing the reliability of permit data in the existing method by statistically analyzing the results. Methods: With the QbD6σ process, the effectiveness and equivalence of the D-antigen quantitative test method were verified with the data of the existing test method and the new test method. Results: Method validation tests are performed based on AQbD. Critical Method Parameters are identified through risk assessment, and single/combined actions are verified by designing and performing tests for Critical Method Parameters (analysis of variance, full factorial design method). Method validation can be effectively accomplished with the QbD6σ process. Conclusion: The use of QbD6σ can be used to achieve satisfactory results for both pharmaceutical companies and regulators by using appropriate statistical analytical methods for method validation as required by regulatory agencies.

유전자 알고리즘과 회귀식을 이용한 오염부하량의 예측 (Estimation of Pollutant Load Using Genetic-algorithm and Regression Model)

  • 박윤식
    • 한국환경농학회지
    • /
    • 제33권1호
    • /
    • pp.37-43
    • /
    • 2014
  • BACKGROUND: Water quality data are collected less frequently than flow data because of the cost to collect and analyze, while water quality data corresponding to flow data are required to compute pollutant loads or to calibrate other hydrology models. Regression models are applicable to interpolate water quality data corresponding to flow data. METHODS AND RESULTS: A regression model was suggested which is capable to consider flow and time variance, and the regression model coefficients were calibrated using various measured water quality data with genetic-algorithm. Both LOADEST and the regression using genetic-algorithm were evaluated by 19 water quality data sets through calibration and validation. The regression model using genetic-algorithm displayed the similar model behaviors to LOADEST. The load estimates by both LOADEST and the regression model using genetic-algorithm indicated that use of a large proportion of water quality data does not necessarily lead to the load estimates with smaller error to measured load. CONCLUSION: Regression models need to be calibrated and validated before they are used to interpolate pollutant loads, as separating water quality data into two data sets for calibration and validation.