• Title/Summary/Keyword: Text Similarity

Search Result 277, Processing Time 0.022 seconds

A Collaborative Filtering System Combined with Users' Review Mining : Application to the Recommendation of Smartphone Apps (사용자 리뷰 마이닝을 결합한 협업 필터링 시스템: 스마트폰 앱 추천에의 응용)

  • Jeon, ByeoungKug;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.2
    • /
    • pp.1-18
    • /
    • 2015
  • Collaborative filtering(CF) algorithm has been popularly used for recommender systems in both academic and practical applications. A general CF system compares users based on how similar they are, and creates recommendation results with the items favored by other people with similar tastes. Thus, it is very important for CF to measure the similarities between users because the recommendation quality depends on it. In most cases, users' explicit numeric ratings of items(i.e. quantitative information) have only been used to calculate the similarities between users in CF. However, several studies indicated that qualitative information such as user's reviews on the items may contribute to measure these similarities more accurately. Considering that a lot of people are likely to share their honest opinion on the items they purchased recently due to the advent of the Web 2.0, user's reviews can be regarded as the informative source for identifying user's preference with accuracy. Under this background, this study proposes a new hybrid recommender system that combines with users' review mining. Our proposed system is based on conventional memory-based CF, but it is designed to use both user's numeric ratings and his/her text reviews on the items when calculating similarities between users. In specific, our system creates not only user-item rating matrix, but also user-item review term matrix. Then, it calculates rating similarity and review similarity from each matrix, and calculates the final user-to-user similarity based on these two similarities(i.e. rating and review similarities). As the methods for calculating review similarity between users, we proposed two alternatives - one is to use the frequency of the commonly used terms, and the other one is to use the sum of the importance weights of the commonly used terms in users' review. In the case of the importance weights of terms, we proposed the use of average TF-IDF(Term Frequency - Inverse Document Frequency) weights. To validate the applicability of the proposed system, we applied it to the implementation of a recommender system for smartphone applications (hereafter, app). At present, over a million apps are offered in each app stores operated by Google and Apple. Due to this information overload, users have difficulty in selecting proper apps that they really want. Furthermore, app store operators like Google and Apple have cumulated huge amount of users' reviews on apps until now. Thus, we chose smartphone app stores as the application domain of our system. In order to collect the experimental data set, we built and operated a Web-based data collection system for about two weeks. As a result, we could obtain 1,246 valid responses(ratings and reviews) from 78 users. The experimental system was implemented using Microsoft Visual Basic for Applications(VBA) and SAS Text Miner. And, to avoid distortion due to human intervention, we did not adopt any refining works by human during the user's review mining process. To examine the effectiveness of the proposed system, we compared its performance to the performance of conventional CF system. The performances of recommender systems were evaluated by using average MAE(mean absolute error). The experimental results showed that our proposed system(MAE = 0.7867 ~ 0.7881) slightly outperformed a conventional CF system(MAE = 0.7939). Also, they showed that the calculation of review similarity between users based on the TF-IDF weights(MAE = 0.7867) leaded to better recommendation accuracy than the calculation based on the frequency of the commonly used terms in reviews(MAE = 0.7881). The results from paired samples t-test presented that our proposed system with review similarity calculation using the frequency of the commonly used terms outperformed conventional CF system with 10% statistical significance level. Our study sheds a light on the application of users' review information for facilitating electronic commerce by recommending proper items to users.

An Analysis of the Pseudo-analytical Thought and Analytical Thought that Students Do in the Process of Algebra Problem Solving (대수 문장제 해결 과정에서 나타나는 擬似(의사) 분석적 사고와 분석적 사고에 대한 분석 - 중학생 대상의 사례 연구 -)

  • Park, Hyun-Jeong;Lee, Chong-Hee
    • Journal of Educational Research in Mathematics
    • /
    • v.17 no.1
    • /
    • pp.67-90
    • /
    • 2007
  • The purpose of this study is to understand students' thinking process in the algebra problem solving, on the base of the works of Vinner(1997a, 1997b). Thus, two middle school students were evaluated in this case study to examine how they think to solve algebra word problems. The following question was considered to analyze the thinking process from the similarity-based perspective by focusing on the process of solving algebra word problems; What is the relationship between similarity and the characteristics of thinking process at the time of successful and unsuccessful problem solving? The following results were obtained by analyzing the success or failure in problem solving based on the characteristics of thinking process and similarity composition. Successful problem solving can be based on pseudo-analytical thought and analytical thought. The former is the rule applied in the process of applying closed formulas that is constructed structural similarity not related with the situations described in the text. The latter means that control and correction occurred in all stages of problem solution. The knowledge needed for solutions was applied with the formulation of open-end formulas that is constructed structural similarity in which memory and modification with the related principles or concepts. In conclusion, the student's perception on the principles involved in a solution is very important in solving algebraic word problems.

  • PDF

Development of An Automatic Classification System for Game Reviews Based on Word Embedding and Vector Similarity (단어 임베딩 및 벡터 유사도 기반 게임 리뷰 자동 분류 시스템 개발)

  • Yang, Yu-Jeong;Lee, Bo-Hyun;Kim, Jin-Sil;Lee, Ki Yong
    • The Journal of Society for e-Business Studies
    • /
    • v.24 no.2
    • /
    • pp.1-14
    • /
    • 2019
  • Because of the characteristics of game software, it is important to quickly identify and reflect users' needs into game software after its launch. However, most sites such as the Google Play Store, where users can download games and post reviews, provide only very limited and ambiguous classification categories for game reviews. Therefore, in this paper, we develop an automatic classification system for game reviews that categorizes reviews into categories that are clearer and more useful for game providers. The developed system converts words in reviews into vectors using word2vec, which is a representative word embedding model, and classifies reviews into the most relevant categories by measuring the similarity between those vectors and each category. Especially, in order to choose the best similarity measure that directly affects the classification performance of the system, we have compared the performance of three representative similarity measures, the Euclidean similarity, cosine similarity, and the extended Jaccard similarity, in a real environment. Furthermore, to allow a review to be classified into multiple categories, we use a threshold-based multi-category classification method. Through experiments on real reviews collected from Google Play Store, we have confirmed that the system achieved up to 95% accuracy.

A Study on the Relationship between Class Similarity and the Performance of Hierarchical Classification Method in a Text Document Classification Problem (텍스트 문서 분류에서 범주간 유사도와 계층적 분류 방법의 성과 관계 연구)

  • Jang, Soojung;Min, Daiki
    • The Journal of Society for e-Business Studies
    • /
    • v.25 no.3
    • /
    • pp.77-93
    • /
    • 2020
  • The literature has reported that hierarchical classification methods generally outperform the flat classification methods for a multi-class document classification problem. Unlike the literature that has constructed a class hierarchy, this paper evaluates the performance of hierarchical and flat classification methods under a situation where the class hierarchy is predefined. We conducted numerical evaluations for two data sets; research papers on climate change adaptation technologies in water sector and 20NewsGroup open data set. The evaluation results show that the hierarchical classification method outperforms the flat classification methods under a certain condition, which differs from the literature. The performance of hierarchical classification method over flat classification method depends on class similarities at levels in the class structure. More importantly, the hierarchical classification method works better when the upper level similarity is less that the lower level similarity.

Similar Contents Recommendation Model Based On Contents Meta Data Using Language Model (언어모델을 활용한 콘텐츠 메타 데이터 기반 유사 콘텐츠 추천 모델)

  • Donghwan Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.1
    • /
    • pp.27-40
    • /
    • 2023
  • With the increase in the spread of smart devices and the impact of COVID-19, the consumption of media contents through smart devices has significantly increased. Along with this trend, the amount of media contents viewed through OTT platforms is increasing, that makes contents recommendations on these platforms more important. Previous contents-based recommendation researches have mostly utilized metadata that describes the characteristics of the contents, with a shortage of researches that utilize the contents' own descriptive metadata. In this paper, various text data including titles and synopses that describe the contents were used to recommend similar contents. KLUE-RoBERTa-large, a Korean language model with excellent performance, was used to train the model on the text data. A dataset of over 20,000 contents metadata including titles, synopses, composite genres, directors, actors, and hash tags information was used as training data. To enter the various text features into the language model, the features were concatenated using special tokens that indicate each feature. The test set was designed to promote the relative and objective nature of the model's similarity classification ability by using the three contents comparison method and applying multiple inspections to label the test set. Genres classification and hash tag classification prediction tasks were used to fine-tune the embeddings for the contents meta text data. As a result, the hash tag classification model showed an accuracy of over 90% based on the similarity test set, which was more than 9% better than the baseline language model. Through hash tag classification training, it was found that the language model's ability to classify similar contents was improved, which demonstrated the value of using a language model for the contents-based filtering.

On a Speech Coding Algorithm for Low Cost Implementation of Voice Telegram System (보이스 전보 시스템 구현을 위한 저가형 음성파형 부호화 알고리즘)

  • 나덕수;민소연;배명진
    • The Journal of the Acoustical Society of Korea
    • /
    • v.19 no.2
    • /
    • pp.101-105
    • /
    • 2000
  • A telegram has been used to transmit the emergency news or celebration message. So, it has been very important media in our life. Although the telegram processing is more and more convenient, on the other hand, the telegram service contains only text message. The voice telegram is that delivering user's voice with text message. So, the voice telegram can be delivered sender's emotions and feelings. However, since voice information contains lots of data, large memory size and high cost processor are needed to deliver itself. In this paper, we proposed a new speech waveform coding method that has low complexity and low cost implementation for the voice telegram system. First, we fixed one basic speech waveform per pitch period and measured the waveform similarity between basic and neighbor speech waveform. Second, if the similarity satisfied threshold values, we compress the neighbor speech waveform with pitch and magnitude value per pitch period and if not, we save speech waveform. When the compression is about 45%, we obtained about 4 point in MOS.

  • PDF

Query Extension of Retrieve System Using Hangul Word Embedding and Apriori (한글 워드임베딩과 아프리오리를 이용한 검색 시스템의 질의어 확장)

  • Shin, Dong-Ha;Kim, Chang-Bok
    • Journal of Advanced Navigation Technology
    • /
    • v.20 no.6
    • /
    • pp.617-624
    • /
    • 2016
  • The hangul word embedding should be performed certainly process for noun extraction. Otherwise, it should be trained words that are not necessary, and it can not be derived efficient embedding results. In this paper, we propose model that can retrieve more efficiently by query language expansion using hangul word embedded, apriori, and text mining. The word embedding and apriori is a step expanding query language by extracting association words according to meaning and context for query language. The hangul text mining is a step of extracting similar answer and responding to the user using noun extraction, TF-IDF, and cosine similarity. The proposed model can improve accuracy of answer by learning the answer of specific domain and expanding high correlation query language. As future research, it needs to extract more correlation query language by analysis of user queries stored in database.

Image Classification Approach for Improving CBIR System Performance (콘텐트 기반의 이미지검색을 위한 분류기 접근방법)

  • Han, Woo-Jin;Sohn, Kyung-Ah
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.41 no.7
    • /
    • pp.816-822
    • /
    • 2016
  • Content-Based image retrieval is a method to search by image features such as local color, texture, and other image content information, which is different from conventional tag or labeled text-based searching. In real life data, the number of images having tags or labels is relatively small, so it is hard to search the relevant images with text-based approach. Existing image search method only based on image feature similarity has limited performance and does not ensure that the results are what the user expected. In this study, we propose and validate a machine learning based approach to improve the performance of the image search engine. We note that when users search relevant images with a query image, they would expect the retrieved images belong to the same category as that of the query. Image classification method is combined with the traditional image feature similarity method. The proposed method is extensively validated on a public PASCAL VOC dataset consisting of 11,530 images from 20 categories.

Development of MSDS Map for Visual Safety Management of Hazardous and Chemical Materials (유해화학물질의 시각적 안전관리를 위한 MSDS 지도 개발)

  • Shin, Myungwoo;Suh, Yongyoon
    • Journal of the Korean Society of Safety
    • /
    • v.34 no.2
    • /
    • pp.48-55
    • /
    • 2019
  • For preventing the accidents generated from the chemical materials, thus far, MSDS (Material Safety Data Sheet) data have been made to notify how to use and manage the hazardous and chemical materials in safety. However, it is difficult for users who handle these materials to understand the MSDS data because they are only listed based on the alphabetical order, not based on the specific factors such as similarity of characteristics. It is limited in representing the types of chemical materials with respect to their characteristics. Thus, in this study, a lots of MSDS data are visualized based on relationships of the characteristics among the chemical materials for supporting safety managers. For this, we used the textmining algorithm which extracts text keywords contained in documents and the Self-Organizing Map (SOM) algorithm which visually addresses textual data information. In the case of Occupational Safety and Health Administration (OSHA) in the United States, the guide texts contained in MSDS documents, which include use information such as reactivity and potential risks of materials, are gathered as the target data. First, using the textmining algorithm, the information of chemicals is extracted from these guide texts. Next, the MSDS map is developed using SOM in terms of similarity of text information of chemical materials. The MSDS map is helpful for effectively classifying chemical materials by mapping prohibited and hazardous substances on the developed the SOM map. As a result, using the MSDS map, it is easy for safety managers to detect prohibited and hazardous substances with respect to the Industrial Safety and Health Act standards.

Method of Extracting the Topic Sentence Considering Sentence Importance based on ELMo Embedding (ELMo 임베딩 기반 문장 중요도를 고려한 중심 문장 추출 방법)

  • Kim, Eun Hee;Lim, Myung Jin;Shin, Ju Hyun
    • Smart Media Journal
    • /
    • v.10 no.1
    • /
    • pp.39-46
    • /
    • 2021
  • This study is about a method of extracting a summary from a news article in consideration of the importance of each sentence constituting the article. We propose a method of calculating sentence importance by extracting the probabilities of topic sentence, similarity with article title and other sentences, and sentence position as characteristics that affect sentence importance. At this time, a hypothesis is established that the Topic Sentence will have a characteristic distinct from the general sentence, and a deep learning-based classification model is trained to obtain a topic sentence probability value for the input sentence. Also, using the pre-learned ELMo language model, the similarity between sentences is calculated based on the sentence vector value reflecting the context information and extracted as sentence characteristics. The topic sentence classification performance of the LSTM and BERT models was 93% accurate, 96.22% recall, and 89.5% precision, resulting in high analysis results. As a result of calculating the importance of each sentence by combining the extracted sentence characteristics, it was confirmed that the performance of extracting the topic sentence was improved by about 10% compared to the existing TextRank algorithm.