Processing math: 100%
  • Title/Summary/Keyword: 자동 평가 지표

Search Result 104, Processing Time 0.029 seconds

Measurement Criteria for Ontology Extraction Tools (온톨로지 자동추출도구의 기능적 성능 평가를 위한 평가지표의 개발 및 적용)

  • Park, Jin-Soo;Cho, Won-Chin;Rho, Sang-Kyu
    • Journal of Intelligence and Information Systems
    • /
    • v.14 no.4
    • /
    • pp.69-87
    • /
    • 2008
  • The Web is evolving toward the Semantic Web. Ontologies are considered as a crucial component of the Semantic Web since it is the backbone of knowledge representation for this Web. However, most of these ontologies are still built manually. Manual building of an ontology is time-consuming activity which requires many resources. Consequently, the need for automatic ontology extraction tools has been increased for the last decade, and many tools have been developed for this purpose. Yet, there is no comprehensive framework for evaluating such tools. In this paper, we proposed a set of criteria for evaluating ontology extraction tools and carried out an experiment on four popular ontology extraction tools (i.e., OntoLT, Text-To-Onto, TERMINAE, and OntoBuilder) using our proposed evaluation framework. The proposed framework can be applied as a useful benchmark when developers want to develop ontology extraction tools.

  • PDF

A Study on Automatic Metrics for Korean Text Abstractive Summarization (한국어 생성 요약 성능 평가 지표 분석 연구)

  • Sehwi Yoon;Youhyun Shin
    • The Transactions of the Korea Information Processing Society
    • /
    • v.13 no.12
    • /
    • pp.691-699
    • /
    • 2024
  • This study aims to analyze and validate automatic evaluation metrics for Korean abstractive summarization. The unique linguistic characteristics of each language require evaluation metrics designed for them, underscoring the importance of research focused on Korean. Research on summarization and its meta-evaluation is extremely limited, especially for Korean. Therefore, by validating reliable automatic evaluation metrics using Korean summarization data, this study contributes to future research on Korean models in the fields of natural language generation. Human evaluation, widely regarded as the most reliable metric, is time-consuming and costly. Thus, research into automatic evaluation metrics holds significant importance for efficiency. In this study, summaries from three models-T5, KoBART,and GPT-3.5 Turbo-were evaluated based on their fluency, consistency, and relevance using 10 Korean documents and their corresponding reference summaries. Correlation coefficients were calculated between human evaluations and automatic metrics for fluency, consistency, and relevance. The results showed that for T5 summaries, the correlation coefficients for consistency and relevance were 0.33 and 0.26, respectively, while for KoBART summaries, the coefficients for fluency and relevance were 0.33 and 0.40, respectively. BERTScore demonstrated the highest correlation, indicating its effectiveness for Korean summaries. Meanwhile, GPT-3.5 Turbo summaries showed significant correlations of 0.23 and 0.17 in consistency and relevance using HaRiM+, a metric developed to detect hallucinations in recent work. Additionally, the correlation analysis by document type revealed that T5 summaries showed high correlations with the BLEU metric for briefing and meeting minutes, KoBART summaries and GPT-3.5 Turbo summaries both demonstrated high correlations with BERTScore for narrative and editorial documents, respectively. These findings emphasize the importance of selecting evaluation metrics tailored to specific document types. Therefore, this study provides a basis for selecting appropriate evaluation metrics tailored to the objectives of specific tasks in future Korean summarization research.

Functional Convergence factor and Usability Evaluation of The Range of Use of The Smart Automatic Urine Disposer (스마트 자동 소변처리기의 기능적 융복합요소와 사용범위의 사용성평가)

  • Kim, Seon-Chil
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.5
    • /
    • pp.31-38
    • /
    • 2021
  • Recently, devices that help the general public's urination activity have been developed, but products that take into account the user's urination habits, device functions, and user accessibility are still insufficient. This study conducted a usability evaluation to find user satisfaction and improvement points of products based on smart automatic urine disposer that have convergence and complex functions such as automatic suction functions. There are 21 indicators used in usability evaluation, and were developed based on safety, operability, and satisfaction. As a result, functional satisfaction was high, but in terms of design, problems to be improved such as handles were presented. Therefore, through this study, it was found that the satisfaction of the automatic urine disposer in terms of performance cannot be representative of the product, and the importance of developing usability evaluation index to find product problems.

Service Level Evaluation Through Measurement Indicators for Public Open Data (공공데이터 개방 평가지표 개발을 통한 현황분석 및 가시화)

  • Kim, Ji-Hye;Cho, Sang-Woo;Lee, Kyung-hee;Cho, Wan-Sup
    • The Journal of Bigdata
    • /
    • v.1 no.1
    • /
    • pp.53-60
    • /
    • 2016
  • Data of central government and local government was collected automatically from the public data portal. And we did the multidimensional analysis based on various perspective like file format and present condition of public data. To complete this work, we constructed Data Warehouse based on the other countries' evaluation index case. Finally, the result from service level evaluation by using multidimensional analysis was used to display each area, establishment, fields.

  • PDF

Effect of Irrigation Automation Using Stem Diameter Variation as an Indicator of Irrigation Timing in Greenhouse Tomato (온실재배 토마토에서 관개시기 진단지표로 경직경 변화를 이용한 관개 자동화 효과)

  • 이변우;신재훈
    • Journal of Bio-Environment Control
    • /
    • v.8 no.4
    • /
    • pp.232-241
    • /
    • 1999
  • The automatic irrigation system using the stem diameter monitoring and the transpiration model for the determination, respectively, of irrigation timing and amount was designed and evaluated for its applicability in pot and field culture of greenhouse tomato. In the pot culture condition, the yield and quality of greenhouse tomato were improved when irrigation was practiced based on the stem diameter monitoring and the transpiration model as compared to the irrigation practice based on soil moisture monitoring. However, the effects were not significant in the field culture condition.

  • PDF

사학연금 재정목표 및 재정지표에 관한 기초연구: 해외 직역연금 사례를 중심으로

  • Min, Gi-Chae;Go, Hye-Jin;Han, Gyeong-Hun;Ju, Bo-Hye;Kim, Ye-Seul
    • Journal of Teachers' Pension
    • /
    • v.2
    • /
    • pp.131-178
    • /
    • 2017
  • 본 연구의 목적은 재정목표와 재정지표에 따라 해외 직역연금을 유형화하고 한국 사학연금의 재정목표와 재정지표 설정을 위한 기초자료를 제시하는 것이다. 주요 분석결과는 국민연금으로부터의 제도적 독립 내지 통합 차원에 따라 직역연금의 재정목표와 재정지표를 유형화할 수 있다는 것이다. 직역연금 독립형(미국 캘리포니아 교직원연금, 대만 사학연금, 캐나다 군인연금)의 경우, 국민연금과 별도의 재정평가 기준이 활용되는 경향이 목도된다. 국민연금-직역연금 통합형(일본 사학연금, 오스트리아 공무원연금)의 경우 전체 국민연금의 재정평가기준 내에서 직역연금의 재정평가가 이루어진다. 해외 직역연금 사례 검토 결과에 기초하여 본 연구에서는 다음의 두 가지를 정책적으로 제언한다. 첫째, 노후소득보장 체계의 전체적(holistic) 맥락에서 한국 사학연금의 명확한 미래 전략을 구상할 것을 제언한다. 둘째, 이러한 미래 전략에 기초하여 한국 사학연금의 재정목표 및 재정지표를 설정하는 것을 정책적으로 제언한다. 사학연금의 재정목표와 재정지표 설정 시 사회보험의 특수성 반영, 재정목표와 재정지표 간 명확한 연계, 재정목표 미달성 보완책으로서 자동조정장치, 재정목표 수정, 구조적 개혁을 고려할 수 있다.

Automatic Identification of Fiducial Marks Existing on Aerial Photographs (항공사진에 포함된 사진 지표의 자동 인식)

  • Cho, Seong-Ik;Bang, Ki-In;Kim, Kyung-Ok
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.10 no.3 s.21
    • /
    • pp.79-87
    • /
    • 2002
  • This paper proposes an approach of automatically identifying the center of fiducial marks using radiometric and geometric characteristics of those marks existing on aerial photographs. Candidate region surrounding a mark, based on radiometric strategy, is determined by producing a bi-level image and by applying morphological operations. Based on geometric strategy, the central location of a mark is determined by applying G filtering and symmetry-enhancement filtering, and by finding peak location of symmetry. Evaluation with respect to 66 cases of sub-images containing a fiducial mark showed that the central location of the mark is determined up to around one pixel difference whit it is compared to a manual inspection.

  • PDF

Scanline Based Metric for Evaluating the Accuracy of Automatic Fracture Survey Methods (자동 균열 조사기법의 정확도 평가를 위한 조사선 기반의 지표 제안)

  • Kim, Jineon;Song, Jae-Joon
    • Tunnel and Underground Space
    • /
    • v.29 no.4
    • /
    • pp.230-242
    • /
    • 2019
  • While various automatic rock fracture survey methods have been researched, the evaluation of the accuracy of these methods raises issues due to the absence of a metric which fully expresses the similarity between automatic and manual fracture maps. Therefore, this paper proposes a geometry similarity metric which is especially designed to determine the overall similarity of fracture maps and to evaluate the accuracy of rock fracture survey methods by a single number. The proposed metric, Scanline Intersection Similarity (SIS), is derived by conducting a large number of scanline surveys upon two fracture maps using Python code. By comparing the frequency of intersections over a large number of scanlines, SIS is able to express the overall similarity between two fracture maps. The proposed metric was compared with Intersection Over Union (IoU) which is a widely used evaluation metric in computer vision. Results showed that IoU is inappropriate for evaluating the geometry similarity of fracture maps because it is overly sensitive to minor geometry differences of thin elongated objects. The proposed metric, on the other hand, reflected macro-geometry differences rather than micro-geometry differences, showing good agreement with human perception. The metric was further applied to evaluate the accuracy of a deep learning-based automatic fracture surveying method which resulted as 0.674 (SIS). However, the proposed metric is currently limited to 2D fracture maps and requires comparison with rock joint parameters such as RQD.

The Influence of Nonlinear Elements on Propulsive Energy Loss Related to Automatic Steering of Ships (자동조타 시스템의 비선형 요소가 선박의 추진 에너지 손실량에 미치는 영향)

  • Lee, Gyoung-Woo;Sohn, Kyoung-Ho;Yoon, Soon-Dong
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 1998.04a
    • /
    • pp.64-74
    • /
    • 1998
  • 본 연구에서는 선박 자동조타시스템의 비선형 요소가 추진 에너지 손실량에 미치는 영향을 정량적으로 평가, 해석하였다. 대양을 자동조타로써 보침 항해중인 선박의 자동조타시스템은 크게 선형 및 비선형요소로 나눌 수있는데, 각 요소들중, 특히 시스템 내부에 인위적으로 또는 불가피하게 설치되어 있는 비선형요소들의 영향은 매우 크다. 본 연구에서는 오토파일럿의 비선형 제어 상수가 전체 시스템의 안정성 또는 추진 에너지 손실의 증감에 미치는 영향을 조사하였다. 본 연구의 수치계산에 이용한 선박은 소형 어선 1척과 대형 광석운반석 1척이며 평균, 풍속 10m/sec 의 대표적 해상상태의 풍·파랑 외란 조건하에서 추지 에너지 손실의 지표인 성능평가지수를 계산하였다. 그리고 수치계산 결과에 대해서도 논하였다.

  • PDF

A Comparison Study of Objective Functions for Automatic Calibration of a Watershed Runoff Continuous Simulation Model (유역 유출 연속모의 모형의 자동 보정을 위한 목적함수 구성에 관한 연구)

  • Ko, Dong-Geun;Lee, Sang-Ho;Kang, Tae-Uk
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2012.05a
    • /
    • pp.279-283
    • /
    • 2012
  • 유역 유출 연속모의 모형은 수자원 계획과 효율적인 물 관리 정책 수립에 중요한 도구가 된다. 유역 유출 연속모의 모형에는 다수의 매개변수가 있으며, 이러한 매개변수는 모형 보정을 통해 추정된다. 연구에서 사용한 모형은 SWMM이며 집합체 혼합 진화 알고리즘으로 자동 보정하였다. 자동 보정에 사용되는 최적화 알고리즘은 목적함수에 따라 상이한 결과를 도출하기도 한다. 이에 따라 본 연구에서는 유역 유출 모형의 자동보정에 적합한 목적함수를 선정하기 위하여 4개의 목적함수를 구성하였고, 밀양댐 유역에 적용하였다. 그리고 목적함수에 따른 자동 보정의 결과를 평가하기 위해 5가지의 평가지표를 활용하였다. 보정의 결과, 모든 목적함수에서 공통적으로 첨두유량의 오차는 다소 크게 발생하였다. 그리고 잔차 절대값의 합이 최소가 되도록 구성한 목적함수가 다른 목적함수에 비해 상대적으로 양호한 결과를 도출하였지만, 목적함수에 따른 큰 차이는 없었다. 또한, 유역 유출 연속모의에서는 유역의 물수지가 중요한 요소이므로 향후, 보다 정확도 높은 유역 유출 연속모의 모형의 자동 보정을 위해서는 첨두유량과 물수지와 관련된 오차를 제어할 수 있는 추가적인 기법이 요구된다.

  • PDF