• Asia pacific journal of information systems
• /
• 제29권4호
• /
• pp.771-788
• /
• 2019

Deep learning techniques such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) show superior performance in text classification than traditional approaches such as Support Vector Machines (SVMs) and Naïve Bayesian approaches. When using CNNs for text classification tasks, word embedding or character embedding is a step to transform words or characters to fixed size vectors before feeding them into convolutional layers. In this paper, we propose a parallel word-level and character-level embedding approach in CNNs for text classification. The proposed approach can capture word-level and character-level patterns concurrently in CNNs. To show the usefulness of proposed approach, we perform experiments with two English and three Korean text datasets. The experimental results show that character-level embedding works better in Korean and word-level embedding performs well in English. Also the experimental results reveal that the proposed approach provides better performance than traditional CNNs with word-level embedding or character-level embedding in both Korean and English documents. From more detail investigation, we find that the proposed approach tends to perform better when there is relatively small amount of data comparing to the traditional embedding approaches.

• 한국컴퓨터정보학회논문지
• /
• 제21권7호
• /
• pp.9-16
• /
• 2016

Collaborative filtering(CF) uses the purchase or item rating history of other users, but does not need additional properties or attributes of users and items. Hence CF is known th be the most successful recommendation technology. But conventional CF approach has some significant weakness, such as the new user problem. In this paper, we propose a approach using word embedding with skip-gram for learning distributed item representations. In particular, we show that this approach can be used to capture precise item for solving the "new user problem." The proposed approach has been tested on the Movielens databases. We compare the performance of the user based CF, item based CF and our approach by observing the change of recommendation results according to the different number of item rating information. The experimental results shows the improvement in our approach in measuring the precision applied to new user problem situations.

• ETRI Journal
• /
• 제44권4호
• /
• pp.599-612
• /
• 2022

This study focuses on improving a word embedding model to enhance the performance of downstream tasks, such as those of dialog systems. To improve traditional word embedding models, such as skip-gram, it is critical to refine the word features and expand the context model. In this paper, we approach the word model from the perspective of subword embedding and attempt to extend the context model by integrating various sentence models. Our proposed sentence model is a subword-based skip-thought model that integrates self-attention and relative position encoding techniques. We also propose a clustering-based dialog model for downstream task verification and evaluate its relationship with the sentence-model-based subword embedding technique. The proposed subword embedding method produces better results than previous methods in evaluating word and sentence similarity. In addition, the downstream task verification, a clustering-based dialog system, demonstrates an improvement of up to 4.86% over the results of FastText in previous research.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제13권1호
• /
• pp.254-276
• /
• 2019

Since the amount of information on the internet is growing rapidly, it is not easy for a user to find relevant information for his/her query. To tackle this issue, the researchers are paying much attention to Document Summarization. The key point in any successful document summarizer is a good document representation. The traditional approaches based on word overlapping mostly fail to produce that kind of representation. Word embedding has shown good performance allowing words to match on a semantic level. Naively concatenating word embeddings makes common words dominant which in turn diminish the representation quality. In this paper, we employ word embeddings to improve the weighting schemes for calculating the Latent Semantic Analysis input matrix. Two embedding-based weighting schemes are proposed and then combined to calculate the values of this matrix. They are modified versions of the augment weight and the entropy frequency that combine the strength of traditional weighting schemes and word embedding. The proposed approach is evaluated on three English datasets, DUC 2002, DUC 2004 and Multilingual 2015 Single-document Summarization. Experimental results on the three datasets show that the proposed model achieved competitive performance compared to the state-of-the-art leading to a conclusion that it provides a better document representation and a better document summary as a result.

• 한국정보시스템학회지:정보시스템연구
• /
• 제30권2호
• /
• pp.57-78
• /
• 2021

Purpose In this study, first, we try to make embedding results that reflect the characteristics of both professional and general documents. In addition, when disparate documents are put together as learning materials for natural language processing, we try to propose a method that can measure the degree of reflection of the characteristics of individual domains in a quantitative way. Approach For this study, the Korean Supreme Court Precedent documents and Korean Wikipedia are selected as specialized documents and general documents respectively. After extracting the most similar word pairs and similarities of unique words observed only in the specialized documents, we observed how those values were changed in the process of embedding with general documents. Findings According to the measurement methods proposed in this study, it was confirmed that the degree of specificity of specialized documents was relaxed in the process of combining with general documents, and that the degree of dissolution could have a positive correlation with the size of general documents.

• 한국음향학회지
• /
• 제39권5호
• /
• pp.468-475
• /
• 2020

단어의 특성을 잘 표현하는 음성 단어 임베딩은 기동어 인식에서 중요한 역할을 한다. 하지만 기동어 인식이 수행되는 환경에서 필연적으로 발생하는 다양한 종류의 잡음으로 인해 음성 단어 임베딩의 표현 능력이 손상될 수 있으며, 인식 성능의 저하를 초래할 수 있다. 본 논문에서는 음성 단어 임베딩에 영향을 줄 수 있는 환경적인 요인을 완화시키는 삼중항 손실 기반의 도메인 적대적 훈련 방식을 제안한다. 잡음 환경에서의 기동어 검출 실험을 통해 제안하는 방식이 기존의 도메인 적대적 훈련 방식을 효과적으로 개선하는 모습을 확인할 수 있었고, 잡음 환경에서의 기동어 검출을 위해 기존에 제안된 다른 방법과의 결합을 통해 제안하는 방식의 확장성을 확인할 수 있었다.

• 정보과학회 논문지
• /
• 제44권3호
• /
• pp.306-313
• /
• 2017

개체명 인식이란 문서 내에서 인명, 기관명, 지명, 시간, 날짜 등 고유한 의미를 가지는 개체명을 추출하여 그 종류를 결정하는 것을 말한다. 최근 개체명 인식 연구에서 가장 우수한 성능을 보여주고 있는 모델은 Bidirectional LSTM CRFs 모델이다. 이러한 LSTM 기반의 딥 러닝 모델은 입력이 되는 단어 표상에 의존적이다. 따라서 입력이 되는 단어를 잘 표현하기 위하여 단어 표상을 확장하는 방법에 대한 연구가 많이 진행되어지고 있다. 본 논문에서는 한국어 개체명 인식을 위하여 Bidirectional LSTM CRFs모델을 사용하고, 그 입력으로 사용되는 단어 표상을 확장하기 위해 사전 학습된 단어 임베딩 벡터, 품사 임베딩 벡터, 음절 기반에서 확장된 단어 임베딩 벡터, 그리고 개체명 사전 자질 벡터를 사용한다. 최종 단어 표상 확장 결과 사전 학습된 단어 임베딩 벡터만 사용한 것 보다 8.05%p의 성능 향상을 보였다.

• International journal of advanced smart convergence
• /
• 제12권3호
• /
• pp.75-88
• /
• 2023

In a specific domain, experts have different understanding of domain knowledge or different purpose of constructing ontology. These will lead to multiple different ontologies in the domain. This phenomenon is called the ontology heterogeneity. For research fields that require cross-ontology operations such as knowledge fusion and knowledge reasoning, the ontology heterogeneity has caused certain difficulties for research. In this paper, we propose a novel ontology matching model that combines word embedding and a concatenated continuous bag-of-words model. Our goal is to improve word vectors and distinguish the semantic similarity and descriptive associations. Moreover, we make the most of textual and structural information from the ontology and external resources. We represent the ontology as a graph and use the SimRank algorithm to calculate the structural similarity. Our approach employs a similarity queue to achieve one-to-many matching results which provide a wider range of insights for subsequent mining and analysis. This enhances and refines the methodology used in ontology matching.

• 정보과학회 논문지
• /
• 제43권6호
• /
• pp.653-661
• /
• 2016

기존의 패턴기반 트리플 생성 시스템은 distant supervision의 가정으로 인해 오류 패턴을 생성하여 트리플 생성 시스템의 성능을 저하시키는 문제점이 있다. 이 문제점을 해결하기 위해 본 논문에서는 패턴과 프로퍼티 사이의 의미 유사도 기반의 패턴 신뢰도를 측정하여 오류 패턴을 제거하는 방법을 제안한다. 의미 유사도 측정은 비지도 학습 방법인 워드임베딩과 워드넷 기반의 어휘 의미 유사도 측정 방법을 결합하여 사용한다. 또한 한국어 패턴과 영어 프로퍼티 사이의 언어 및 어휘 불일치 문제를 해결하기 위해 정준 상관 분석과 사전 기반의 번역을 사용한다. 실험 결과에 따르면 제안한 의미 유사도 기반의 패턴 신뢰도 측정 방법이 기존의 방법보다 10% 높은 정확률의 트리플 집합을 생성하여, 트리플 생성 성능 향상을 증명하였다.

• 지식경영연구
• /
• 제21권1호
• /
• pp.117-136
• /
• 2020

In the studies for the recommender systems which solve the information overload problem of users, the use of transactional data has been continuously tried. Especially, because the firms can easily obtain transactional data along with the development of IoT technologies, transaction-based recommender systems are recently used in various areas. However, the use of transactional data has limitations that it is hard to reflect domain knowledge and they do not directly show user preferences for individual items. Therefore, in this study, we propose a method applying the word embedding in the transaction-based recommender system to reflect preference differences among users and domain knowledge. Our approach is based on SAR, which shows high performance in the recommender systems, and we improved its components by using FastText, one of the word embedding techniques. Experimental results show that the reflection of domain knowledge and preference difference has a significant effect on the performance of recommender systems. Therefore, we expect our study to contribute to the improvement of the transaction-based recommender systems and to suggest the expansion of data used in the recommender system.