• Title/Summary/Keyword: Automatic Metric

Search Result 50, Processing Time 0.024 seconds

Scanline Based Metric for Evaluating the Accuracy of Automatic Fracture Survey Methods (자동 균열 조사기법의 정확도 평가를 위한 조사선 기반의 지표 제안)

  • Kim, Jineon;Song, Jae-Joon
    • Tunnel and Underground Space
    • /
    • v.29 no.4
    • /
    • pp.230-242
    • /
    • 2019
  • While various automatic rock fracture survey methods have been researched, the evaluation of the accuracy of these methods raises issues due to the absence of a metric which fully expresses the similarity between automatic and manual fracture maps. Therefore, this paper proposes a geometry similarity metric which is especially designed to determine the overall similarity of fracture maps and to evaluate the accuracy of rock fracture survey methods by a single number. The proposed metric, Scanline Intersection Similarity (SIS), is derived by conducting a large number of scanline surveys upon two fracture maps using Python code. By comparing the frequency of intersections over a large number of scanlines, SIS is able to express the overall similarity between two fracture maps. The proposed metric was compared with Intersection Over Union (IoU) which is a widely used evaluation metric in computer vision. Results showed that IoU is inappropriate for evaluating the geometry similarity of fracture maps because it is overly sensitive to minor geometry differences of thin elongated objects. The proposed metric, on the other hand, reflected macro-geometry differences rather than micro-geometry differences, showing good agreement with human perception. The metric was further applied to evaluate the accuracy of a deep learning-based automatic fracture surveying method which resulted as 0.674 (SIS). However, the proposed metric is currently limited to 2D fracture maps and requires comparison with rock joint parameters such as RQD.

Automatic Method for Contrast Enhancement of Natural Color Images

  • Lal, Shyam;Narasimhadhan, A. V.;Kumar, Rahul
    • Journal of Electrical Engineering and Technology
    • /
    • v.10 no.3
    • /
    • pp.1233-1243
    • /
    • 2015
  • The contrast enhancement is great challenge in the image processing when images are suffering from poor contrast problem. Therefore, in order to overcome this problem an automatic method is proposed for contrast enhancement of natural color images. The proposed method consist of two stages: in first stage lightness component in YIQ color space is normalized by sigmoid function after the adaptive histogram equalization is applied on Y component and in second stage automatic color contrast enhancement algorithm is applied on output of the first stage. The proposed algorithm is tested on different NASA color images, hyperspectral color images and other types of natural color images. The performance of proposed algorithm is evaluated and compared with the other existing contrast enhancement algorithms in terms of colorfulness metric and color enhancement factor. The higher values of colorfulness metric and color enhancement factor imply that the visual quality of the enhanced image is good. Simulation results demonstrate that proposed algorithm provides higher values of colorfulness metric and color enhancement factor as compared to other existing contrast enhancement algorithms. The proposed algorithm also provides better visual enhancement results as compared with the other existing contrast enhancement algorithms.

Collaborative Similarity Metric Learning for Semantic Image Annotation and Retrieval

  • Wang, Bin;Liu, Yuncai
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.7 no.5
    • /
    • pp.1252-1271
    • /
    • 2013
  • Automatic image annotation has become an increasingly important research topic owing to its key role in image retrieval. Simultaneously, it is highly challenging when facing to large-scale dataset with large variance. Practical approaches generally rely on similarity measures defined over images and multi-label prediction methods. More specifically, those approaches usually 1) leverage similarity measures predefined or learned by optimizing for ranking or annotation, which might be not adaptive enough to datasets; and 2) predict labels separately without taking the correlation of labels into account. In this paper, we propose a method for image annotation through collaborative similarity metric learning from dataset and modeling the label correlation of the dataset. The similarity metric is learned by simultaneously optimizing the 1) image ranking using structural SVM (SSVM), and 2) image annotation using correlated label propagation, with respect to the similarity metric. The learned similarity metric, fully exploiting the available information of datasets, would improve the two collaborative components, ranking and annotation, and sequentially the retrieval system itself. We evaluated the proposed method on Corel5k, Corel30k and EspGame databases. The results for annotation and retrieval show the competitive performance of the proposed method.

Three-Stage Framework for Unsupervised Acoustic Modeling Using Untranscribed Spoken Content

  • Zgank, Andrej
    • ETRI Journal
    • /
    • v.32 no.5
    • /
    • pp.810-818
    • /
    • 2010
  • This paper presents a new framework for integrating untranscribed spoken content into the acoustic training of an automatic speech recognition system. Untranscribed spoken content plays a very important role for under-resourced languages because the production of manually transcribed speech databases still represents a very expensive and time-consuming task. We proposed two new methods as part of the training framework. The first method focuses on combining initial acoustic models using a data-driven metric. The second method proposes an improved acoustic training procedure based on unsupervised transcriptions, in which word endings were modified by broad phonetic classes. The training framework was applied to baseline acoustic models using untranscribed spoken content from parliamentary debates. We include three types of acoustic models in the evaluation: baseline, reference content, and framework content models. The best overall result of 18.02% word error rate was achieved with the third type. This result demonstrates statistically significant improvement over the baseline and reference acoustic models.

Decision on the Optimal Photographing Angle and Overlapping Ratio of Non-metric Cameras for Development of Automatic Image Stitching System (영상집성 자동화 시스템 개발을 위한 비측량용 카메라의 최적 촬영각 및 중복도 결정)

  • Kim, Dae Sung;Shin, Sang Chul
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.21 no.2
    • /
    • pp.117-123
    • /
    • 2013
  • This study focuses on the determination of optimal photographing angle and overlapping ratio for automatic image stitching system using a non-metric camera module with motorized head. Photographing overlap was calculated considering the angle of view on different kinds of lenses and moving angle of motorized head per each option, and optimal photographing angle and overlapping ratio could be determined through the experimental result using the operating time, data volume and performance of image stitching. Through this experiment, we could find that it was effective to take a picture with 3636 of interval(33.82% of overlap) in vertical direction and 2424 or 3030 of interval(36.51% or 20.63% of overlap) in horizontal direction using 35mm lens for automatic image stitching system.

Framework for evaluating code generation ability of large language models

  • Sangyeop Yeo;Yu-Seung Ma;Sang Cheol Kim;Hyungkook Jun;Taeho Kim
    • ETRI Journal
    • /
    • v.46 no.1
    • /
    • pp.106-117
    • /
    • 2024
  • Large language models (LLMs) have revolutionized various applications in natural language processing and exhibited proficiency in generating programming code. We propose a framework for evaluating the code generation ability of LLMs and introduce a new metric, pass-ratio@n, which captures the granularity of accuracy according to the pass rate of test cases. The framework is intended to be fully automatic to handle the repetitive work involved in generating prompts, conducting inferences, and executing the generated codes. A preliminary evaluation focusing on the prompt detail, problem publication date, and difficulty level demonstrates the successful integration of our framework with the LeetCode coding platform and highlights the applicability of the pass-ratio@n metric.

An Automatic Construction Approach of State Diagram from Class Operations with Pre/Post Conditions (클래스 연산의 선행/후행 조건에 바탕을 둔 클래스의 상태 다이어그램 자동 구성 기법)

  • Lee, Kwang-Min;Bae, Jung-Ho;Chae, Heung-Seok
    • The KIPS Transactions:PartD
    • /
    • v.16D no.4
    • /
    • pp.527-540
    • /
    • 2009
  • State diagrams describe the dynamic behavior of an individual object as a number of states and transitions between these states. In this paper, we propose an automated technique to the generation of a state diagram from class operations with pre/post conditions. And I also develop a supporting tool, SDAG (State Diagram Automatic Generation tool). Additionally, we propose a complexity metric and a state diagram generation approach concerning types of each operation for decreasing complexity of generated state diagram.

Development of Automatic Tool for Software Metrics Analysis for Railway Signaling System (열차제어시스템 소프트웨어 Metrics 분석 자동화 도구 개발)

  • Hwang, Jong-Gyu;Jo, Hyun-Jeong;Kim, Yong-Kyu
    • Journal of the Korean Society for Railway
    • /
    • v.12 no.4
    • /
    • pp.450-456
    • /
    • 2009
  • In accordance with the development of recent computer technology, the dependency of railway signaling system on the computer software is being increased further, and accordingly, the testing for the safety and reliability of railway signaling system software became more important This thesis suggested automated an analysis tool for S/W metrics on this railway signaling system, and presented its result of implementation. The analysis items in the implemented tool had referred to the international standards in relation to the software for railway system, such as IEC61508 and IEC 62279. This automated analysis tool for railway signaling system can be utilized at the assessment stage for railway signaling system software also, and it is anticipated that it can be utilized usefully at the software development stage also.

A Study on Automatic Metrics for Korean Text Abstractive Summarization (한국어 생성 요약 성능 평가 지표 분석 연구)

  • Sehwi Yoon;Youhyun Shin
    • The Transactions of the Korea Information Processing Society
    • /
    • v.13 no.12
    • /
    • pp.691-699
    • /
    • 2024
  • This study aims to analyze and validate automatic evaluation metrics for Korean abstractive summarization. The unique linguistic characteristics of each language require evaluation metrics designed for them, underscoring the importance of research focused on Korean. Research on summarization and its meta-evaluation is extremely limited, especially for Korean. Therefore, by validating reliable automatic evaluation metrics using Korean summarization data, this study contributes to future research on Korean models in the fields of natural language generation. Human evaluation, widely regarded as the most reliable metric, is time-consuming and costly. Thus, research into automatic evaluation metrics holds significant importance for efficiency. In this study, summaries from three models-T5, KoBART,and GPT-3.5 Turbo-were evaluated based on their fluency, consistency, and relevance using 10 Korean documents and their corresponding reference summaries. Correlation coefficients were calculated between human evaluations and automatic metrics for fluency, consistency, and relevance. The results showed that for T5 summaries, the correlation coefficients for consistency and relevance were 0.33 and 0.26, respectively, while for KoBART summaries, the coefficients for fluency and relevance were 0.33 and 0.40, respectively. BERTScore demonstrated the highest correlation, indicating its effectiveness for Korean summaries. Meanwhile, GPT-3.5 Turbo summaries showed significant correlations of 0.23 and 0.17 in consistency and relevance using HaRiM+, a metric developed to detect hallucinations in recent work. Additionally, the correlation analysis by document type revealed that T5 summaries showed high correlations with the BLEU metric for briefing and meeting minutes, KoBART summaries and GPT-3.5 Turbo summaries both demonstrated high correlations with BERTScore for narrative and editorial documents, respectively. These findings emphasize the importance of selecting evaluation metrics tailored to specific document types. Therefore, this study provides a basis for selecting appropriate evaluation metrics tailored to the objectives of specific tasks in future Korean summarization research.

Quality Improvement of B-spline Surfaces through Fairing of Data Points (측정점의 순정을 통한 B-스플라인 곡면 품질의 개선)

  • 흥석용;이현찬
    • Korean Journal of Computational Design and Engineering
    • /
    • v.6 no.1
    • /
    • pp.40-47
    • /
    • 2001
  • In reverse engineering, existing products are digitized fur the computer modeling. Using the digitized data, surfaces are modeled for new products. However, in the digitizing process measuring errors or deviations can be happened often in practice. Thus, it is important to adjust such errors or deviations during the computer modeling. To adjust the errors, fairing of the modeled surfaces is performed. In this paper, we present a surface fairing algorithm based on various fairness metrics. Fairness metrics can be discrete. We adopt discrete metrics for fairing given 3D point set. The fairness metrics include discrete principal curvatures. In this paper, automatic fairing process is proposed for fairing given 3D point sets for surfaces. The process uses various fairness criteria so that it is adequate to adopt designers'intents.

  • PDF