• Title/Summary/Keyword: word dictionary

Search Result 276, Processing Time 0.025 seconds

Parting Lyrics Emotion Classification using Word2Vec and LSTM (Word2Vec과 LSTM을 활용한 이별 가사 감정 분류)

  • Lim, Myung Jin;Park, Won Ho;Shin, Ju Hyun
    • Smart Media Journal
    • /
    • v.9 no.3
    • /
    • pp.90-97
    • /
    • 2020
  • With the development of the Internet and smartphones, digital sound sources are easily accessible, and accordingly, interest in music search and recommendation is increasing. As a method of recommending music, research using melodies such as pitch, tempo, and beat to classify genres or emotions is being conducted. However, since lyrics are becoming one of the means of expressing human emotions in music, the role of the lyrics is increasing, so a study of emotion classification based on lyrics is needed. Therefore, in this thesis, we analyze the emotions of the farewell lyrics in order to subdivide the farewell emotions based on the lyrics. After constructing an emotion dictionary by vectoriziong the similarity between words appearing in the parting lyrics through Word2Vec learning, we propose a method of classifying parting lyrics emotions using Word2Vec and LSTM, which classify lyrics by similar emotions by learning lyrics using LSTM.

Bi-directional Maximal Matching Algorithm to Segment Khmer Words in Sentence

  • Mao, Makara;Peng, Sony;Yang, Yixuan;Park, Doo-Soon
    • Journal of Information Processing Systems
    • /
    • v.18 no.4
    • /
    • pp.549-561
    • /
    • 2022
  • In the Khmer writing system, the Khmer script is the official letter of Cambodia, written from left to right without a space separator; it is complicated and requires more analysis studies. Without clear standard guidelines, a space separator in the Khmer language is used inconsistently and informally to separate words in sentences. Therefore, a segmented method should be discussed with the combination of the future Khmer natural language processing (NLP) to define the appropriate rule for Khmer sentences. The critical process in NLP with the capability of extensive data language analysis necessitates applying in this scenario. One of the essential components in Khmer language processing is how to split the word into a series of sentences and count the words used in the sentences. Currently, Microsoft Word cannot count Khmer words correctly. So, this study presents a systematic library to segment Khmer phrases using the bi-directional maximal matching (BiMM) method to address these problematic constraints. In the BiMM algorithm, the paper focuses on the Bidirectional implementation of forward maximal matching (FMM) and backward maximal matching (BMM) to improve word segmentation accuracy. A digital or prefix tree of data structure algorithm, also known as a trie, enhances the segmentation accuracy procedure by finding the children of each word parent node. The accuracy of BiMM is higher than using FMM or BMM independently; moreover, the proposed approach improves dictionary structures and reduces the number of errors. The result of this study can reduce the error by 8.57% compared to FMM and BFF algorithms with 94,807 Khmer words.

Choosing preferable labels for the Japanese translation of the Human Phenotype Ontology

  • Ninomiya, Kota;Takatsuki, Terue;Kushida, Tatsuya;Yamamoto, Yasunori;Ogishima, Soichi
    • Genomics & Informatics
    • /
    • v.18 no.2
    • /
    • pp.23.1-23.6
    • /
    • 2020
  • The Human Phenotype Ontology (HPO) is the de facto standard ontology to describe human phenotypes in detail, and it is actively used, particularly in the field of rare disease diagnoses. For clinicians who are not fluent in English, the HPO has been translated into many languages, and there have been four initiatives to develop Japanese translations. At the Biomedical Linked Annotation Hackathon 6 (BLAH6), a rule-based approach was attempted to determine the preferable Japanese translation for each HPO term among the candidates developed by the four approaches. The relationship between the HPO and Mammalian Phenotype translations was also investigated, with the eventual goal of harmonizing the two translations to facilitate phenotype-based comparisons of species in Japanese through cross-species phenotype matching. In order to deal with the increase in the number of HPO terms and the need for manual curation, it would be useful to have a dictionary containing word-by-word correspondences and fixed translation phrases for English word order. These considerations seem applicable to HPO localization into other languages.

A Joint Statistical Model for Word Spacing and Spelling Error Correction Simultaneously (띄어쓰기 및 철자 오류 동시교정을 위한 통계적 모델)

  • Noh, Hyung-Jong;Cha, Jeong-Won;Lee, GaryGeun-Bae
    • Journal of KIISE:Software and Applications
    • /
    • v.34 no.2
    • /
    • pp.131-139
    • /
    • 2007
  • In this paper, we present a preprocessor which corrects word spacing errors and spelling correction errors simultaneously. The proposed expands noisy-channel model so that it corrects both errors in colloquial style sentences effectively, while preprocessing algorithms have limitations because they correct each error separately. Using Eojeol transition pattern dictionary and statistical data such as n-gram and Jaso transition probabilities, it minimizes the usage of dictionaries and produces the corrected candidates effectively. In experiments we did not get satisfactory results at current stage, we noticed that the proposed methodology has the utility by analyzing the errors. So we expect that the preprocessor will function as an effective error corrector for general colloquial style sentence by doing more improvements.

Resolving Multi-Translatable Verbs Japanese-TO-Korean Machine Translation

  • Kim Jung-In;Lee Kang-Hyuk
    • Journal of Korea Multimedia Society
    • /
    • v.8 no.6
    • /
    • pp.790-797
    • /
    • 2005
  • It is well-known that there are many similarities between Japanese and Korean language. For example, the order of words and the nature of the grammatical conjugation of both languages are almost the same. Another similarity is the frequent omission of the subject from a sentence. Moreover, both languages have honorific expressions and the identical concept for expressing nouns in terms of Chinese characters. Using these similarities, we have developed a word-to-word translation system which does away with any deep level analysis of syntactic and semantic structures of the two languages. If we use these similarities, the direct translation method is superior to the internal language translation method or transfer-based translation method. Although the MT system based on the direct translation method is more easily developed than the ones based on other methods, it may have a lot of difficulties when it tries to select the appropriate target word from ambiguous source verbs. In this paper, we propose a new algorithm to extract the meaning of substantives and to make use of the order of the extracted meaning. We could select $86.5\%$ appropriate verbs in the sample sentences from IPAL-verb-dictionary. $13.5\%$ indicates the cases in which we could not distinguish the meaning of substantives. We are convinced, however, that the succeeding rate can be increased by getting rid of the meaning of verbs thatare not used so often.

  • PDF

Identifying Similar Overseas Patent Using Word2Vec-Based Semantic Text Analytics (Word2Vec 학습을 통한 의미 기반 해외 유사 특허 검색 방안)

  • Paek, Minji;Kim, Namgyu
    • Journal of Information Technology Services
    • /
    • v.17 no.2
    • /
    • pp.129-142
    • /
    • 2018
  • Recently, the number of patent applications have been increasing rapidly every year as the importance of protecting intellectual property rights becomes more important. Patents must be inventive and have novelty. Especially, the novelty implies that the corresponding invention is not the same as the previous invention. To confirm the novelty, prior art search must be conducted before and after the application. The target of prior art search should include not only Korean patents but also foreign patents. Search of foreign patents should be supported by multilingual search techniques. However, a dictionary-based naive approach shows a limitation because some technical concepts are represented in different terms according to each nation. For example, a Korean term and a Japanese term may not be synonym even though they represent the same technical concept. In this paper, we propose a new method to map semantic similarity between technical terms in Korean patents and Japanese patents. To investigate different representations in each nation for the same technical concept, we identified and analyzed pairs of patents those are mutually connected with priority claim relationship. By performing an experiment with real-world data, we showed that our approach can reveal semantically similar technical terms in other language successfully.

A Study on the Characteristic of Logomark in Apparel Brand - Focused on Unisex Casual Brand - (의류 브랜드 로고마크의 특성에 관한 연구 - 유니섹스 캐주얼 브랜드를 중심으로 -)

  • Lee, Min-Gyung;Rha, Soo-Im
    • The Research Journal of the Costume Culture
    • /
    • v.13 no.5 s.58
    • /
    • pp.833-843
    • /
    • 2005
  • The purpose of this study was to investigate the characteristic of logomark in unisex casual apparel brand. For this study, first 36 unisex casual apparel brands were selected from the Dictionary of Fashion Brand, second analyzed the common word showing in them. Third, the logomark of the unisex casual apparel brand were classified into two types according to the typeface character of the logomark, there were serif typeface, sans-serif typeface. Fourth, analyzed the relationship between the typeface image of logomark and brand concept. The results of the study were following : First, the common word that used the most frequently in brand concept were investigated and the order of common word was reasonable, comfort or natural, practical, modern, traditional and basic. Second, The unisex casual apparel brand used the most frequently the sans-serif typeface that represents the images of simple, modern and active sense in the typeface of logomark. Third, the unisex apparel brands used the most frequently English as brand name among the various languages. Fourth, the unisex casual apparel brands were lanuched mostly except several of them after 1990.

  • PDF

Design of environmental technology search system using synonym dictionary (유의어 사전 기반 환경기술 검색 시스템 설계)

  • XIANGHUA, PIAO;HELIN, YIN;Gu, Yeong Hyeon;Yoo, Seong Joon
    • Proceedings of the Korean Society of Broadcast Engineers Conference
    • /
    • 2020.07a
    • /
    • pp.582-586
    • /
    • 2020
  • 국가기후기술정보시스템은 국내 환경기술과 국외의 수요기술 정보를 제공하는 검색 시스템이다. 그러나 기존의 시스템은 유사한 뜻을 가진 단일 단어와 복수 단어들을 모두 식별하지 못하기에 유의어를 입력했을 경우 검색 결과가 다르다. 이런 문제점을 해결하기 위해 본 연구에서는 유의어 사전을 기반으로한 환경기술 검색 시스템을 제안한다. 이 시스템은 Word2vec 모델과 HDBSCAN(Hierarchical Density-Based Spatial Clustering of Application with Noise) 알고리즘을 이용해 유의어 사전을 구축한다. Word2vec 모델을 이용해 한국어와 영어 위키백과 코퍼스에 대해 형태소 분석을 진행한 후 단일 단어와 복수 단어를 포함한 단어를 추출하고 벡터화를 진행한다. 그 다음 HDBSCAN 알고리즘을 이용해 벡터화된 단어를 군집화 해주고 유의어를 추출한다. 기존의 Word2vec 모델이 모든 단어 간의 거리를 계산하고 유의어를 추출하는 과정과 대비하면 시간이 단축되는 역할을 한다. 추출한 유의어를 통합해 유의어 사전을 구축한다. 국가기후기술정보시스템에서 제공하는 국내외 기술정보, 기술정보 키워드와 구축한 유의어 사전을 Multi-filter를 제공하는 Elasticsearch에 적용해 최종적으로 유의어를 식별할 수 있는 환경기술 검색 시스템을 제안한다.

  • PDF

Efficient Keyword Extraction from Social Big Data Based on Cohesion Scoring

  • Kim, Hyeon Gyu
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.10
    • /
    • pp.87-94
    • /
    • 2020
  • Social reviews such as SNS feeds and blog articles have been widely used to extract keywords reflecting opinions and complaints from users' perspective, and often include proper nouns or new words reflecting recent trends. In general, these words are not included in a dictionary, so conventional morphological analyzers may not detect and extract those words from the reviews properly. In addition, due to their high processing time, it is inadequate to provide analysis results in a timely manner. This paper presents a method for efficient keyword extraction from social reviews based on the notion of cohesion scoring. Cohesion scores can be calculated based on word frequencies, so keyword extraction can be performed without a dictionary when using it. On the other hand, their accuracy can be degraded when input data with poor spacing is given. Regarding this, an algorithm is presented which improves the existing cohesion scoring mechanism using the structure of a word tree. Our experiment results show that it took only 0.008 seconds to extract keywords from 1,000 reviews in the proposed method while resulting in 15.5% error ratio which is better than the existing morphological analyzers.

(A Method to Classify and Recognize Spelling Changes between Morphemes of a Korean Word) (한국어 어절의 철자변화 현상 분류와 인식 방법)

  • 김덕봉
    • Journal of KIISE:Software and Applications
    • /
    • v.30 no.5_6
    • /
    • pp.476-486
    • /
    • 2003
  • There is no explicit spelling change information in part-of-speech tagged corpora of Korean. It causes some difficulties in acquiring the data to study Korean morphology, i.e. automatically in constructing a dictionary for morphological analysis and systematically in collecting the phenomena of the spelling changes from the corpora. To solve this problem, this paper presents a method to recognize spelling changes between morphemes of a Korean word in tagged corpora, only using a string matching, without using a dictionary and phonological rules. This method not only has an ability to robustly recognize the spelling changes because it doesn't use any phonological rules, but also can be implemented with few cost. This method has been experimented with a large tagged corpus of Korean, and recognized the 100% of spelling changes in the corpus with accuracy.