• Title/Summary/Keyword: Data quality diagnosis

Search Result 416, Processing Time 0.028 seconds

Proposal of Public Data Quality Management Level Evaluation Domain Rule Mapping Model

  • Jeong, Ha-Na;Kim, Jae-Woong;Chung, Young-Suk
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.12
    • /
    • pp.189-195
    • /
    • 2022
  • The Korean government has made it a major national task to contribute to the revitalization of the creative economy, such as creating new industries and jobs, by encouraging the private opening and utilization of public data. The Korean government is promoting public data quality improvement through activities such as conducting public data quality management level evaluation for high-quality public data retention. However, there is a difference in diagnosis results depending on the understanding and data expertise of users of the public data quality diagnosis tool. Therefore, it is difficult to ensure the accuracy of the diagnosis results. This paper proposes a public data quality management level evaluation domain rule mapping model applicable to validation diagnosis among the data quality diagnosis standards. This increases the stability and accuracy of public data quality diagnosis.

A Study on the Domain Discrimination Model of CSV Format Public Open Data

  • Ha-Na Jeong;Jae-Woong Kim;Young-Suk Chung
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.12
    • /
    • pp.129-136
    • /
    • 2023
  • The government of the Republic of Korea is conducting quality management of public open data by conducting a public data quality management level evaluation. Public open data is provided in various open formats such as XML, JSON, and CSV, with CSV format accounting for the majority. When diagnosing the quality of public open data in CSV format, the quality diagnosis manager determines and diagnoses the domain for each field based on the field name and data within the field of the public open data file. However, it takes a lot of time because quality diagnosis is performed on large amounts of open data files. Additionally, in the case of fields whose meaning is difficult to understand, the accuracy of quality diagnosis is affected by the quality diagnosis person's ability to understand the data. This paper proposes a domain discrimination model for public open data in CSV format using field names and data distribution statistics to ensure consistency and accuracy so that quality diagnosis results are not influenced by the capabilities of the quality diagnosis person in charge, and to support shortening of diagnosis time. As a result of applying the model in this paper, the correct answer rate was about 77%, which is 2.8% higher than the file format open data diagnostic tool provided by the Ministry of Public Administration and Security. Through this, we expect to be able to improve accuracy when applying the proposed model to diagnosing and evaluating the quality management level of public data.

A Study of Big Data Domain Automatic Classification Using Machine Learning (머신러닝을 이용한 빅데이터 도메인 자동 판별에 관한 연구)

  • Kong, Seongwon;Hwang, Deokyoul
    • The Journal of Bigdata
    • /
    • v.3 no.2
    • /
    • pp.11-18
    • /
    • 2018
  • This study is a study on domain automatic classification for domain - based quality diagnosis which is a key element of big data quality diagnosis. With the increase of the value and utilization of Big Data and the rise of the Fourth Industrial Revolution, the world is making efforts to create new value by utilizing big data in various fields converged with IT such as law, medical, and finance. However, analysis based on low-reliability data results in critical problems in both the process and the result, and it is also difficult to believe that judgments based on the analysis results. Although the need of highly reliable data has also increased, research on the quality of data and its results have been insufficient. The purpose of this study is to shorten the work time to automizing the domain classification work which was performed from manually to using machine learning in the domain - based quality diagnosis, which is a key element of diagnostic evaluation for improving data quality. Extracts information about the characteristics of the data that is stored in the database and identifies the domain, and then featurize it, and automizes the domain classification using machine learning. We will use it for big data quality diagnosis and contribute to quality improvement.

Proposal of diagnosis rule mapping model to support public data quality diagnosis (공공데이터 품질진단 지원을 위한 진단규칙 매핑모델 제안)

  • Jeong, Ha-Na;Kim, Jae-Woong;Lee, Yun-Yeol;Chae, Yi-Geun;Chung, Young-Suk
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2022.07a
    • /
    • pp.127-128
    • /
    • 2022
  • 정부는 공공데이터 개방을 통해 신산업, 일자리 창출 등 경제 활성화를 위한 도구로 활용하는 것을 목표로 한다. 정부는 고품질의 공공데이터 보유를 위하여 품질 개선 활동을 통해 공공데이터 품질 향상을 진행하고 있다. 그러나 공공데이터 품질관리 수준 진단을 진행하는 담당자의 데이터에 대한 전문성과 이해도에 따라 품질진단 결과에 격차가 발생하여 진단 결과의 신뢰성을 보장하기 어렵다. 본 논문은 공공데이터의 원활한 품질진단 지원을 위해 품질진단규칙 매핑 모델을 제안하여 공공데이터 품질진단의 안정성과 신뢰성을 높인다.

  • PDF

A Study on Automation of Big Data Quality Diagnosis Using Machine Learning (머신러닝을 이용한 빅데이터 품질진단 자동화에 관한 연구)

  • Lee, Jin-Hyoung
    • The Journal of Bigdata
    • /
    • v.2 no.2
    • /
    • pp.75-86
    • /
    • 2017
  • In this study, I propose a method to automate the method to diagnose the quality of big data. The reason for automating the quality diagnosis of Big Data is that as the Fourth Industrial Revolution becomes a issue, there is a growing demand for more volumes of data to be generated and utilized. Data is growing rapidly. However, if it takes a lot of time to diagnose the quality of the data, it can take a long time to utilize the data or the quality of the data may be lowered. If you make decisions or predictions from these low-quality data, then the results will also give you the wrong direction. To solve this problem, I have developed a model that can automate diagnosis for improving the quality of Big Data using machine learning which can quickly diagnose and improve the data. Machine learning is used to automate domain classification tasks to prevent errors that may occur during domain classification and reduce work time. Based on the results of the research, I can contribute to the improvement of data quality to utilize big data by continuing research on the importance of data conversion, learning methods for unlearned data, and development of classification models for each domain.

  • PDF

Development of Power Quality Management System with Power Quality Diagnosis Functions

  • Chung Il-Yop;Won Dong-Jun;Ahn Seon-Ju;Kim Joong-Moon;Moon Seung-Il;Seo Jang-Cheol;Choe Jong-Woong;Jang Gil-Soo
    • Journal of Electrical Engineering and Technology
    • /
    • v.1 no.1
    • /
    • pp.28-34
    • /
    • 2006
  • Recently, in accordance with the development of IT technology, it is prevalent for power quality monitors to be connected to each other via networks and share their data because such networks provide system-wide insights to customers concerning power quality. Those systems can alarm and display power quality events for the convenience of customers. However, if a power quality event occurs, it is difficult for customers to determine its cause and solution because the systems do not provide appropriate power quality diagnosis functions. The power quality management system presented in this paper has been developed to provide customers with various power quality diagnosis functions so that they can cope well with power quality problems with the right measure in the right place. This paper presents the structure and functions of the developed power quality management system and shows some results of the power diagnosis functions.

Quality Diagnosis and Improvement of Fisheries Census Statistic (어업조사통계의 품질진단과 개선에 관한 연구)

  • Pyo, Hee-Dong;Kim, Jong-Chun
    • Journal of Fisheries and Marine Sciences Education
    • /
    • v.22 no.4
    • /
    • pp.553-565
    • /
    • 2010
  • The paper aims to evaluate the quality of fisheries census statistic and to provide some desirable directions and improvements for the future fisheries census, conducted by the Government. For the quality diagnosis of fisheries census statistic, specific processes of fisheries census and statistical qualities of each dimension are surveyed and evaluated by a Government's practician, two external examiners and a research group. Results show that census design, data analysis and quality control are evaluated relatively low in specific processes, and accessibility and comparability are evaluated relatively lower than relevance, accuracy, timeliness and consistency in statistical qualities. For minimizing the sampling errors, the probability proportion method should be employed in sampling methods from currently simple sampling method. In addition, fisheries census statistic is desirable to include and compare with those of different countries for consumer oriented data system.

Quality Diagnosis of Library-Related Open Government Data: Focused on Book Details API of Data for Library (도서관 공공데이터의 품질에 관한 연구: 도서관 정보나루의 도서 상세 조회 API를 중심으로)

  • Yang, Suwan
    • Journal of the Korean Society for information Management
    • /
    • v.37 no.4
    • /
    • pp.181-206
    • /
    • 2020
  • With the popularization of open government data, Library-related open government data is also open and utilized to the public. The purpose of this paper is to diagnose the quality of library-related open government data and propose improvement measures to enhance the quality based on the diagnosis result. As a result of diagnosing the completeness of the data, a number of blanks are identified in the bibliographic elements essential for identifying and searching a book. As a result of diagnosing the accuracy of the data, the bibliographic elements that are not compliant with the data schema have been identified. Based on the result of data quality diagnosis, this study suggested improving the data collection procedure, establishing data set schema, providing details on data collection and data processing, and publishing raw data.

Development of Construction Model of Disease Classification on Clinical Diagnosis in Ophthalmology (임상진단명에 따른 질병분류체계 구축모형 개발 - 안과를 대상으로 -)

  • Suh, Jin-Sook;Shin, Hee-Young;Kee, Chang-Won
    • Quality Improvement in Health Care
    • /
    • v.10 no.2
    • /
    • pp.204-215
    • /
    • 2003
  • Background : ICD-10 Classification, which is used domestically as well as internationally, has limited use in the clinical practice since it is developed for at disease statistics and epidemiology. Therefore, the purposes of this study were to improve the quality of diagnosis by constructing a new disease classification based on the diagnoses doctors currently make in the clinical setting and connecting this classification with OCS and EMR, and to meet the demands of doctors for high quality medical study data in medical research. Methods : The specialists in each ophthalmic subfield collected clinical diagnoses and abbreviations based on the ophthalmology textbooks and confirmed the classifications. Total number of clinical diagnoses collected was totaled 672, for which ideal diagnoses had been selected and a new model of disease classification model in connection with ICD-10 was constructed. The constructed classification of clinical diagnoses consisted of six steps: the first step was the classification by ophthalmic subspecialty field; the second to fifth steps were the detailed classification by each specialty field; the sixth step was the classification by site. Results : After introducing the new disease classification, research on the use and a pre-post comparison was conducted. The result from the research on the use of the clinical diagnoses in inpatient and outpatient care has shown a gradually increasing tendency. From the pre-post comparison of EMR discharge summary diagnoses, the result demonstrated that the diagnosis was stated correctly and in detail. Since the diagnosis was stated correctly, code classification became correct as well, which makes it possible to construct high quality medical DB. Conclusion : This construction of clinical diagnoses provides the medical team with high quality medical information. It is also expected to increase the accuracy and efficiency of service in the department of medical record and department of insurance investigation. In the future, if hospitals wish to construct a classification of clinical diagnosis and a standard proposal of clinical diagnosis is presented by a medical society, the standardization of diagnosis seems to be possible.

  • PDF

Analysis of Healthcare Quality Indicators using Data Mining and Development of a Decision Support System (데이터마이닝을 이용한 의료의 질 측정지표 분석 및 의사결정지원시스템 개발)

  • Kim, Hye Sook;Chae, Young-Moon;Tark, Kwan-Chul;Park, Hyun-Ju;Ho, Seung-Hee
    • Quality Improvement in Health Care
    • /
    • v.8 no.2
    • /
    • pp.186-207
    • /
    • 2001
  • Background : This study presented an analysis of healthcare quality indicators using data mining and a development of decision support system for quality improvement. Method : Specifically, important factors influencing the key quality indicators were identified using a decision tree method for data mining based on 8,405 patients who discharged from a medical center during the period between December 1, 2000 and January 31, 2001. In addition, a decision support system was developed to analyze and monitor trends of these quality indicators using a Visual Basic 6.0. Guidelines and tutorial for quality improvement activities were also included in the system. Result : Among 12 selected quality indicators, decision tree analysis was performed for 3 indicators ; unscheduled readmission due to the same or related condition, unscheduled return to intensive care unit, and inpatient mortality which have a volume bigger than 100 cases during the period. The optimum range of target group in healthcare quality indicators were identified from the gain chart. Important influencing factors for these 3 indicators were: diagnosis, attribute of the disease, and age of the patient in unscheduled returns to ICU group ; and length of stay, diagnosis, and belonging department in inpatient mortality group. Conclusion : We developed a decision support system through analysis of healthcare quality indicators and data mining technique which can be effectively implemented for utilization review and quality management in a healthcare organization. In the future, further number of quality indicators should be developed to effectively support a hospital-wide Continuous Quality Improvement activity. Through these endevours, a decision support system can be developed and the newly developed decision support system should be well integrated with the hospital Order Communication System to support concurrent review, utilization review, quality and risk management.

  • PDF