• Proceedings of the Korean Information Science Society Conference
• /
• 2007.10c
• /
• pp.119-124
• /
• 2007

인터넷 웹사이트의 급격한 증가와 함께 이용자도 증가하고 있으며, 이용 목적은 주로 자료검색과 조회서비스 이다. 조회 요청이 많을수록 질의의 증가를 야기하며, 데이터베이스 서버의 질의 분석(Parse), 질의 실행 계획(Query Execution Plan)을 과도하게 발생시킨다. 즉 데이터베이스 서버에서 처리하는 작업량의 과부하로 인하여 병목현상을 초래한다. 데이터베이스 서버의 조회를 위한 질의처리량을 감소시키는 작업이 필요하다. 그리고 조회 대상이 데이터는 웹사이트에서 자주 갱신되지 않거나, 데이터가 주기적으로 갱신되는 특징이 있다. 이 데이터를 대상으로 데이터베이스 캐쉬 엔진을 구성하면 데이터베이스 서버의 과부하률 해소 할 수 있다. 본 논문에서는 유사 질의 매칭 기반 데이터베이스 캐친 엔진을 설계하고 구현한다. 유사 질의 매칭 기반으로 하여 적중률을 높여 데이터베이스 병목현상을 해결하여, 검색서비스에 더욱 효과적일 것으로 사료되며, 웹사이트의 성능 향상을 기대한다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.568-570
• /
• 2022

객체 유사도를 판별하는 기술은 정보 처리의 여러 분야에서 응용되고 있다. 본 연구에서는 현장 자연어 텍스트 데이터셋과 딥러닝 모델을 이용하여 챗봇 등에서 응용되는 데이터 유사성을 판별하고, 해당 모델의 성능을 측정해보았다.

• Journal of Intelligence and Information Systems
• /
• v.29 no.1
• /
• pp.27-40
• /
• 2023

With the increase in the spread of smart devices and the impact of COVID-19, the consumption of media contents through smart devices has significantly increased. Along with this trend, the amount of media contents viewed through OTT platforms is increasing, that makes contents recommendations on these platforms more important. Previous contents-based recommendation researches have mostly utilized metadata that describes the characteristics of the contents, with a shortage of researches that utilize the contents' own descriptive metadata. In this paper, various text data including titles and synopses that describe the contents were used to recommend similar contents. KLUE-RoBERTa-large, a Korean language model with excellent performance, was used to train the model on the text data. A dataset of over 20,000 contents metadata including titles, synopses, composite genres, directors, actors, and hash tags information was used as training data. To enter the various text features into the language model, the features were concatenated using special tokens that indicate each feature. The test set was designed to promote the relative and objective nature of the model's similarity classification ability by using the three contents comparison method and applying multiple inspections to label the test set. Genres classification and hash tag classification prediction tasks were used to fine-tune the embeddings for the contents meta text data. As a result, the hash tag classification model showed an accuracy of over 90% based on the similarity test set, which was more than 9% better than the baseline language model. Through hash tag classification training, it was found that the language model's ability to classify similar contents was improved, which demonstrated the value of using a language model for the contents-based filtering.

• Annual Conference on Human and Language Technology
• /
• 2019.10a
• /
• pp.265-271
• /
• 2019

본 논문은 어휘가 비슷한 문장들을 효과적으로 분류하는 BERT 기반 유사 문장 분류기의 학습 자료 구성 방법을 제안한다. 기존의 유사 문장 분류기는 문장의 의미와 상관 없이 각 문장에서 출현한 어휘의 유사도를 기준으로 분류하였다. 이는 학습 자료 내의 유사 문장 쌍들이 유사하지 않은 문장 쌍들보다 어휘 유사도가 높기 때문이다. 따라서, 본 논문은 어휘 유사도가 높은 유사 의미 문장 쌍들과 어휘 유사도가 높지 않은 의미 문장 쌍들을 학습 자료에 추가하여 BERT 유사 문장 분류기를 학습하여 전체 분류 성능을 크게 향상시켰다. 이는 문장의 의미를 결정짓는 단어들과 그렇지 않은 단어들을 유사 문장 분류기가 학습하였기 때문이다. 제안하는 학습 데이터 구축 방법을 기반으로 학습된 BERT 유사 문장 분류기들의 학습된 self-attention weight들을 비교 분석하여 BERT 내부에서 어떤 변화가 발생하였는지 확인하였다.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.26 no.4
• /
• pp.591-597
• /
• 2022

Recently, many studies have been done on the data augmentation technique as a way to efficiently build datasets. Among them, a representative data augmentation technique is a method of utilizing Generative Adversarial Network (GAN), which generates data similar to real data by competitively learning generators and discriminators. However, when learning GAN, there are cases where a broken pixel image occurs among similar data generated according to the environment and progress, which cannot be used as a dataset and causes an increase in learning time. In this paper, an algorithm was developed to select these damaged images by analyzing the histogram of image data generated during the GAN learning process, and as a result of comparing them with the images generated in the existing GAN, the ratio of the damaged images was reduced by 33.3 times(3,330%).

• Journal of the Korean Data and Information Science Society
• /
• v.26 no.2
• /
• pp.367-376
• /
• 2015

Today, big data has become a hot keyword in that big data may be defined as collection of data sets so huge and complex that it becomes difficult to process by traditional methods. Clustering method is to identify the information in a big database by assigning a set of objects into the clusters so that the objects in the same cluster are more similar to each other clusters. The similarity measures being used in the cluster analysis may be classified into various types depending on the nature of the data. In this paper, we computed upper and lower limits for probability interestingness measure based similarity measures without marginal probability such as Yule I and II, Michael, Digby, Baulieu, and Dispersion measure. And we compared these measures by real data and simulated experiment. By Warrens (2008), Coefficients with the same quantities in the numerator and denominator, that are bounded, and are close to each other in the ordering, are likely to be more similar. Thus, results on bounds provide means of classifying various measures. Also, knowing which coefficients are similar provides insight into the stability of a given algorithm.

• The KIPS Transactions:PartD
• /
• v.11D no.4
• /
• pp.823-834
• /
• 2004

In this research, the similarity search algorithms are provided for large video data streams. A video stream that consists of a number of frames can be expressed by a sequence in the multidimensional data space, by representing each frame with a multidimensional vector By analyzing various characteristics of the sequence, it is partitioned into multiple video segments and clusters which are represented by hyper-rectangles. Using the hyper-rectangles of video segments and clusters, similarity functions between two video streams are defined, and two similarity search algorithms are proposed based on the similarity functions algorithms by hyper-rectangles and by representative frames. The former is an algorithm that guarantees the correctness while the latter focuses on the efficiency with a slight sacrifice of the correctness Experiments on different types of video streams and synthetically generated stream data show the strength of our proposed algorithms.

• Journal of the Korea Convergence Society
• /
• v.12 no.7
• /
• pp.53-60
• /
• 2021

LOD-based recommender systems usually leverage the data available within LOD datasets, such as DBpedia, in order to recommend items(movies, books, music) to the end users. These systems use a semantic similarity algorithm that calculates the degree of matching between pairs of Linked Data resources. In this paper, we proposed a new approach to measuring semantic distance in an LOD-based recommender system by assigning weights converted from user ratings to links in the LOD graph. The semantic distance measurement model proposed in this paper is based on a processing step in which a graph is personalized to a user through weight calculation and a method of applying these weights to LDSD. The Experimental results showed that the proposed method showed higher accuracy compared to other similar methods, and it contributed to the improvement of similarity by expanding the range of semantic distance measurement of the recommender system. As future work, we aim to analyze the impact on the model using different methods of LOD-based similarity measurement.

• Journal of the Korea Society of Computer and Information
• /
• v.26 no.5
• /
• pp.47-53
• /
• 2021

Sparse ratings data hinder reliable similarity computation between users, which degrades the performance of memory-based collaborative filtering techniques for recommender systems. Many works in the literature have been developed for solving this data sparsity problem, where the most simple and representative ones are the methods of utilizing Jaccard index. This index reflects the number of commonly rated items between two users and is mostly integrated into traditional similarity measures to compute similarity more accurately between the users. However, such integration is very straightforward with no consideration of the degree of data sparsity. This study suggests a novel idea of applying different similarity measures depending on the numeric value of Jaccard index between two users. Performance experiments are conducted to obtain optimal values of the parameters used by the proposed method and evaluate it in comparison with other relevant methods. As a result, the proposed demonstrates the best and comparable performance in prediction and recommendation accuracies.

• Journal of IKEEE
• /
• v.22 no.3
• /
• pp.816-821
• /
• 2018

Recently, machine learning algorithms based on artificial neural networks started to be used widely as classifiers in the field of EEG research for emotion analysis and disease diagnosis. When a machine learning model is used to classify EEG data, if training data is composed of only data having similar characteristics, classification performance may be deteriorated when applied to data of another group. In this paper, we propose a method to construct training data set by selecting several groups of data using semi-supervised learning algorithm to improve these problems. We then compared the performance of the two models by training the model with a training data set consisting of data with similar characteristics to the training data set constructed using the proposed method.