• Title/Summary/Keyword: Recall and Precision

Search Result 715, Processing Time 0.027 seconds

Passage Retrieval based on Tracing Topic Continuity and Transition by Using Field-Associated Term (분야연상어를 이용한 화제의 계속성과 전환성을 추적하는 단락분할 방법)

  • Lee, Sang-Kon
    • The KIPS Transactions:PartB
    • /
    • v.10B no.1
    • /
    • pp.57-66
    • /
    • 2003
  • We propose a technique to extract a relevant passage from text collection based on field-associated terms since they tries to concentrate relevant text to users query. Documents are supposed to be managed as a whole without any segmentation into small pieces, but the method presented is independent upon any text-embedded auxiliary information, and is based on topic continuity and transition. For users needs-relative sentences or passages, we present a passage retrieval techniques by using occurrence frequency of a field-associated term to delimit text, that is likely to be relevant to a particular topic, considering continuity and transition within topic flowing in text. We evaluate 50 Japanese documents and verify the usefulness with 82% for average precision and 63% for recall.

Road Surface Damage Detection Based on Semi-supervised Learning Using Pseudo Labels (수도 레이블을 활용한 준지도 학습 기반의 도로노면 파손 탐지)

  • Chun, Chanjun;Ryu, Seung-Ki
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.18 no.4
    • /
    • pp.71-79
    • /
    • 2019
  • By using convolutional neural networks (CNNs) based on semantic segmentation, road surface damage detection has being studied. In order to generate the CNN model, it is essential to collect the input and the corresponding labeled images. Unfortunately, such collecting pairs of the dataset requires a great deal of time and costs. In this paper, we proposed a road surface damage detection technique based on semi-supervised learning using pseudo labels to mitigate such problem. The model is updated by properly mixing labeled and unlabeled datasets, and compares the performance against existing model using only labeled dataset. As a subjective result, it was confirmed that the recall was slightly degraded, but the precision was considerably improved. In addition, the $F_1-score$ was also evaluated as a high value.

Method of Extracting the Topic Sentence Considering Sentence Importance based on ELMo Embedding (ELMo 임베딩 기반 문장 중요도를 고려한 중심 문장 추출 방법)

  • Kim, Eun Hee;Lim, Myung Jin;Shin, Ju Hyun
    • Smart Media Journal
    • /
    • v.10 no.1
    • /
    • pp.39-46
    • /
    • 2021
  • This study is about a method of extracting a summary from a news article in consideration of the importance of each sentence constituting the article. We propose a method of calculating sentence importance by extracting the probabilities of topic sentence, similarity with article title and other sentences, and sentence position as characteristics that affect sentence importance. At this time, a hypothesis is established that the Topic Sentence will have a characteristic distinct from the general sentence, and a deep learning-based classification model is trained to obtain a topic sentence probability value for the input sentence. Also, using the pre-learned ELMo language model, the similarity between sentences is calculated based on the sentence vector value reflecting the context information and extracted as sentence characteristics. The topic sentence classification performance of the LSTM and BERT models was 93% accurate, 96.22% recall, and 89.5% precision, resulting in high analysis results. As a result of calculating the importance of each sentence by combining the extracted sentence characteristics, it was confirmed that the performance of extracting the topic sentence was improved by about 10% compared to the existing TextRank algorithm.

Detecting Foreign Objects in Chest X-Ray Images using Artificial Intelligence (인공 지능을 이용한 흉부 엑스레이 이미지에서의 이물질 검출)

  • Chang-Hwa Han
    • Journal of the Korean Society of Radiology
    • /
    • v.17 no.6
    • /
    • pp.873-879
    • /
    • 2023
  • This study explored the use of artificial intelligence(AI) to detect foreign bodies in chest X-ray images. Medical imaging, especially chest X-rays, plays a crucial role in diagnosing diseases such as pneumonia and lung cancer. With the increase in imaging tests, AI has become an important tool for efficient and fast diagnosis. However, images can contain foreign objects, including everyday jewelry like buttons and bra wires, which can interfere with accurate readings. In this study, we developed an AI algorithm that accurately identifies these foreign objects and processed the National Institutes of Health chest X-ray dataset based on the YOLOv8 model. The results showed high detection performance with accuracy, precision, recall, and F1-score all close to 0.91. Despite the excellent performance of AI, the study solved the problem that foreign objects in the image can distort the reading results, emphasizing the innovative role of AI in radiology and its reliability based on accuracy, which is essential for clinical implementation.

Detection and Prediction of Subway Failure using Machine Learning (머신러닝을 이용한 지하철 고장 탐지 및 예측)

  • Kuk-Kyung Sung
    • Advanced Industrial SCIence
    • /
    • v.2 no.4
    • /
    • pp.11-16
    • /
    • 2023
  • The subway is a means of public transportation that plays an important role in the transportation system of modern cities. However, congestion often occurs due to sudden breakdowns and system outages, causing inconvenience. Therefore, in this paper, we conducted a study on failure prediction and prevention using machine learning to efficiently operate the subway system. Using UC Irvine's MetroPT-3 dataset, we built a subway breakdown prediction model using logistic regression. The model predicted the non-failure state with a high accuracy of 0.991. However, precision and recall are relatively low, suggesting the possibility of error in failure prediction. The ROC_AUC value is 0.901, indicating that the model can classify better than random guessing. The constructed model is useful for stable operation of the subway system, but additional research is needed to improve performance. Therefore, in the future, if there is a lot of learning data and the data is well purified, failure can be prevented by pre-inspection through prediction.

Fashion Category Oversampling Automation System

  • Minsun Yeu;Do Hyeok Yoo;SuJin Bak
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.1
    • /
    • pp.31-40
    • /
    • 2024
  • In the realm of domestic online fashion platform industry the manual registration of product information by individual business owners leads to inconvenience and reliability issues, especially when dealing with simultaneous registrations of numerous product groups. Moreover, bias is significantly heightened due to the low quality of product images and an imbalance in data quantity. Therefore, this study proposes a ResNet50 model aimed at minimizing data bias through oversampling techniques and conducting multiple classifications for 13 fashion categories. Transfer learning is employed to optimize resource utilization and reduce prolonged learning times. The results indicate improved discrimination of up to 33.4% for data augmentation in classes with insufficient data compared to the basic convolution neural network (CNN) model. The reliability of all outcomes is underscored by precision and affirmed by the recall curve. This study is suggested to advance the development of the domestic online fashion platform industry to a higher echelon.

Semantic Process Retrieval with Similarity Algorithms (유사도 알고리즘을 활용한 시맨틱 프로세스 검색방안)

  • Lee, Hong-Joo;Klein, Mark
    • Asia pacific journal of information systems
    • /
    • v.18 no.1
    • /
    • pp.79-96
    • /
    • 2008
  • One of the roles of the Semantic Web services is to execute dynamic intra-organizational services including the integration and interoperation of business processes. Since different organizations design their processes differently, the retrieval of similar semantic business processes is necessary in order to support inter-organizational collaborations. Most approaches for finding services that have certain features and support certain business processes have relied on some type of logical reasoning and exact matching. This paper presents our approach of using imprecise matching for expanding results from an exact matching engine to query the OWL(Web Ontology Language) MIT Process Handbook. MIT Process Handbook is an electronic repository of best-practice business processes. The Handbook is intended to help people: (1) redesigning organizational processes, (2) inventing new processes, and (3) sharing ideas about organizational practices. In order to use the MIT Process Handbook for process retrieval experiments, we had to export it into an OWL-based format. We model the Process Handbook meta-model in OWL and export the processes in the Handbook as instances of the meta-model. Next, we need to find a sizable number of queries and their corresponding correct answers in the Process Handbook. Many previous studies devised artificial dataset composed of randomly generated numbers without real meaning and used subjective ratings for correct answers and similarity values between processes. To generate a semantic-preserving test data set, we create 20 variants for each target process that are syntactically different but semantically equivalent using mutation operators. These variants represent the correct answers of the target process. We devise diverse similarity algorithms based on values of process attributes and structures of business processes. We use simple similarity algorithms for text retrieval such as TF-IDF and Levenshtein edit distance to devise our approaches, and utilize tree edit distance measure because semantic processes are appeared to have a graph structure. Also, we design similarity algorithms considering similarity of process structure such as part process, goal, and exception. Since we can identify relationships between semantic process and its subcomponents, this information can be utilized for calculating similarities between processes. Dice's coefficient and Jaccard similarity measures are utilized to calculate portion of overlaps between processes in diverse ways. We perform retrieval experiments to compare the performance of the devised similarity algorithms. We measure the retrieval performance in terms of precision, recall and F measure? the harmonic mean of precision and recall. The tree edit distance shows the poorest performance in terms of all measures. TF-IDF and the method incorporating TF-IDF measure and Levenshtein edit distance show better performances than other devised methods. These two measures are focused on similarity between name and descriptions of process. In addition, we calculate rank correlation coefficient, Kendall's tau b, between the number of process mutations and ranking of similarity values among the mutation sets. In this experiment, similarity measures based on process structure, such as Dice's, Jaccard, and derivatives of these measures, show greater coefficient than measures based on values of process attributes. However, the Lev-TFIDF-JaccardAll measure considering process structure and attributes' values together shows reasonably better performances in these two experiments. For retrieving semantic process, we can think that it's better to consider diverse aspects of process similarity such as process structure and values of process attributes. We generate semantic process data and its dataset for retrieval experiment from MIT Process Handbook repository. We suggest imprecise query algorithms that expand retrieval results from exact matching engine such as SPARQL, and compare the retrieval performances of the similarity algorithms. For the limitations and future work, we need to perform experiments with other dataset from other domain. And, since there are many similarity values from diverse measures, we may find better ways to identify relevant processes by applying these values simultaneously.

Semantics Environment for U-health Service driven Naive Bayesian Filtering for Personalized Service Recommendation Method in Digital TV (디지털 TV에서 시멘틱 환경의 유헬스 서비스를 위한 나이브 베이지안 필터링 기반 개인화 서비스 추천 방법)

  • Kim, Jae-Kwon;Lee, Young-Ho;Kim, Jong-Hun;Park, Dong-Kyun;Kang, Un-Gu
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.8
    • /
    • pp.81-90
    • /
    • 2012
  • For digital TV, the recommendation of u-health personalized service of semantic environment should be done after evaluating individual physical condition, illness and health condition. The existing recommendation method of u-health personalized service of semantic environment had low user satisfaction because its recommendation was dependent on ontology for analyzing significance. We propose the personalized service recommendation method based on Naive Bayesian Classifier for u-health service of semantic environment in digital TV. In accordance with the proposed method, the condition data is inferred by using ontology, and the transaction is saved. By applying naive bayesian classifier that uses preference information, the service is provided after inferring based on user preference information and transaction formed from ontology. The service inferred based on naive bayesian classifier shows higher precision and recall ratio of the contents recommendation rather than the existing method.

Calculation of a Threshold for Decision of Similar Features in Different Spatial Data Sets (이종의 공간 데이터 셋에서 매칭 객체 판별을 위한 임계값 산출)

  • Kim, Jiyoung;Huh, Yong;Yu, Kiyun;Kim, Jung Ok
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.31 no.1
    • /
    • pp.23-28
    • /
    • 2013
  • The process of a feature matching for two different spatial data sets is similar to the process of classification as a binary class such as matching or non-matching. In this paper, we calculated a threshold by applying an equal error rate (EER) which is widely used in biometrics that classification is a main topic into spatial data sets. In a process of discriminating what's a matching or what's not, a precision and a recall is changed and a trade-off appears between these indexes because the number of matching pairs is changed when a threshold is changed progressively. This trade-off point is EER, that is, threshold. To the result of applying this method into training data, a threshold is estimated at 0.802 of a value of shape similarity. By applying the estimated threshold into test data, F-measure that is a evaluation index of matching method is highly value, 0.940. Therefore we confirmed that an accurate threshold is calculated by EER without person intervention and this is appropriate to matching different spatial data sets.

Validation Technique for Class Name Postfixes Based on the Machine Learning of Class Properties (클래스 특성 기계학습에 기반한 클래스 이름의 접미사 검증 기법)

  • Lee, Hongseok;Lee, Junha;Lee, Illo;Park, Soojin;Park, Sooyong
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.4 no.6
    • /
    • pp.247-252
    • /
    • 2015
  • As software has gotten bigger in magnitude and the complexity of software has been increased, the maintenance has gained in-creasing attention for its significant impact on the cost. Identifiers have an impact on more than 90 percent of the readability which accounts for a majority portion of the maintenance activities. For this reason, the existing works focus on domain-specific features based on identifiers. However, their approaches have a limitation when either a class name does not reflect the intention of its context or a class naming is incorrect. Therefore, this paper suggests a series of class name validation process by extracting properties of classes, building learning model by applying a decision tree technique of machine learning, and generating a validation report containing the list of recommendable postfixes of classes to be validated. To evaluate this, four open source projects are selected and indicators such as precision, recall, and ROC curve present the value of this work when it comes to five specific postfixes including functional information on class names.