• 제목/요약/키워드: Korean text analysis

Search Result 2,042, Processing Time 0.031 seconds

Investigations on Techniques and Applications of Text Analytics (텍스트 분석 기술 및 활용 동향)

  • Kim, Namgyu;Lee, Donghoon;Choi, Hochang;Wong, William Xiu Shun
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.42 no.2
    • /
    • pp.471-492
    • /
    • 2017
  • The demand and interest in big data analytics are increasing rapidly. The concepts around big data include not only existing structured data, but also various kinds of unstructured data such as text, images, videos, and logs. Among the various types of unstructured data, text data have gained particular attention because it is the most representative method to describe and deliver information. Text analysis is generally performed in the following order: document collection, parsing and filtering, structuring, frequency analysis, and similarity analysis. The results of the analysis can be displayed through word cloud, word network, topic modeling, document classification, and semantic analysis. Notably, there is an increasing demand to identify trending topics from the rapidly increasing text data generated through various social media. Thus, research on and applications of topic modeling have been actively carried out in various fields since topic modeling is able to extract the core topics from a huge amount of unstructured text documents and provide the document groups for each different topic. In this paper, we review the major techniques and research trends of text analysis. Further, we also introduce some cases of applications that solve the problems in various fields by using topic modeling.

Inferring Undiscovered Public Knowledge by Using Text Mining Analysis and Main Path Analysis: The Case of the Gene-Protein 'brings_about' Chains of Pancreatic Cancer (텍스트마이닝과 주경로 분석을 이용한 미발견 공공 지식 추론 - 췌장암 유전자-단백질 유발사슬의 경우 -)

  • Ahn, Hyerim;Song, Min;Heo, Go Eun
    • Journal of the Korean BIBLIA Society for library and Information Science
    • /
    • v.26 no.1
    • /
    • pp.217-231
    • /
    • 2015
  • This study aims to infer the gene-protein 'brings_about' chains of pancreatic cancer which were referred to in the pancreatic cancer related researches by constructing the gene-protein interaction network of pancreatic cancer. The chains can help us uncover publicly unknown knowledge that would develop as empirical studies for investigating the cause of pancreatic cancer. In this study, we applied a novel approach that grafts text mining and the main path analysis into Swanson's ABC model for expanding intermediate concepts to multi-levels and extracting the most significant path. We carried out text mining analysis on the full texts of the pancreatic cancer research papers published during the last ten-year period and extracted the gene-protein entities and relations. The 'brings_about' network was established with bio relations represented by bio verbs. We also applied main path analysis to the network. We found the main direct 'brings_about' path of pancreatic cancer which includes 14 nodes and 13 arcs. 9 arcs were confirmed as the actual relations emerged on the related researches while the other 4 arcs were arisen in the network transformation process for main path analysis. We believe that our approach to combining text mining analysis with main path analysis can be a useful tool for inferring undiscovered knowledge in the situation where either a starting or an ending point is unknown.

Document Layout Analysis Based on Fuzzy Energy Matrix

  • Oh, KangHan;Kim, SooHyung
    • International Journal of Contents
    • /
    • v.11 no.2
    • /
    • pp.1-8
    • /
    • 2015
  • In this paper, we describe a novel method for document layout analysis that is based on a Fuzzy Energy Matrix (FEM). A FEM is a two-dimensional matrix that contains the likelihood of text and non-text and is generated through the use of Fuzzy theory. The key idea is to define an Energy map for the document to categorize text and non-text. The proposed mechanism is designed for execution with a low-resolution document image, and hence our method has a fast processing speed. The proposed method has been tested on public ICDAR 2009 datasets to conduct a comparison against other state-of-the-art methods, and it was also tested with Korean documents. The results of the experiment indicate that this scheme achieves superior segmentation accuracy, in terms of both precision and recall, and also requires less time for computation than other state-of-the-art document image analysis methods.

A Public Open Civil Complaint Data Analysis Model to Improve Spatial Welfare for Residents - A Case Study of Community Welfare Analysis in Gangdong District - (거주민 공간복지 향상을 위한 공공 개방 민원 데이터 분석 모델 - 강동구 공간복지 분석 사례를 중심으로 -)

  • Shin, Dongyoun
    • Journal of KIBIM
    • /
    • v.13 no.3
    • /
    • pp.39-47
    • /
    • 2023
  • This study aims to introduce a model for enhancing community well-being through the utilization of public open data. To objectively assess abstract notions of residential satisfaction, text data from complaints is analyzed. By leveraging accessible public data, costs related to data collection are minimized. Initially, relevant text data containing civic complaints is collected and refined by removing extraneous information. This processed data is then combined with meaningful datasets and subjected to topic modeling, a text mining technique. The insights derived are visualized using Geographic Information System (GIS) and Application Programming Interface (API) data. The efficacy of this analytical model was demonstrated in the Godeok/Gangil area. The proposed methodology allows for comprehensive analysis across time, space, and categories. This flexible approach involves incorporating specific public open data as needed, all within the overarching framework.

Investigation of the Possibility of Research on Medical Classics Applying Text Mining - Focusing on the Huangdi's Internal Classic - (텍스트마이닝(Text mining)을 활용한 한의학 원전 연구의 가능성 모색 -『황제내경(黃帝內經)』에 대한 적용례를 중심으로 -)

  • Bae, Hyo-jin;Kim, Chang-eop;Lee, Choong-yeol;Shin, Sang-won;Kim, Jong-hyun
    • Journal of Korean Medical classics
    • /
    • v.31 no.4
    • /
    • pp.27-46
    • /
    • 2018
  • Objectives : In this paper, we investigated the applicability of text mining to Korean Medical Classics and suggest that researchers of Medical Classics utilize this methodology. Methods : We applied text mining to the Huangdi's internal classic, a seminal text of Korean Medicine, and visualized networks which represent connectivity of terms and documents based on vector similarity. Then we compared this outcome to the prior knowledge generated through conventional qualitative analysis and examined whether our methodology could accurately reflect the keyword of documents, clusters of terms, and relationships between documents. Results : In the term network, we confirmed that Qi played a key role in the term network and that the theory development based on relativity between Yin and Yang was reflected. In the document network, Suwen and Lingshu are quite distinct from each other due to their differences in description form and topic. Also, Suwen showed high similarity between adjacent chapters. Conclusions : This study revealed that text mining method could yield a significant discovery which corresponds to prior knowledge about Huangdi's internal classic. Text mining can be used in a variety of research fields covering medical classics, literatures, and medical records. In addition, visualization tools can also be utilized for educational purposes.

- For the Development of Inquiring, integrated Science Curricular Materials - The Comparison and Analysis of Inquiry Activity between "The FAST Program" and "The Secondary Science Books" (탐구적 통합 과학 교재 개발을 위한, "FAST program"과 "중등 과학 교과서"의 탐구 활동 비교 분석)

  • Son, Yeon-A;Lee, Hack-Dong
    • Journal of The Korean Association For Science Education
    • /
    • v.14 no.1
    • /
    • pp.45-57
    • /
    • 1994
  • The purpose of this study is to verify whether the FAST program is the Inquiry Science Curricular Materials, through the Comparison and Analysis of Inquiry Activities between the FAST program and our Secondary Science Books. The results of this study are as follows ; 1. FAST has 226 tasks of the Inquiry Activities, which is analyzed over two times than our text. 2. In level one, FAST holds the parts of Synthesizing Results and Evaluation, Hypothesizing and Designing an Experiment but u.ese aren't found in our text. 3. In level two, our text is analyzed No Discussion 72.2%, Demonstrating or Verifying the Content of the Text 82%, but FAST has Discussion Guided 81.8%, and isn't found any tesk of Demonstrating or Verifying the Content of the text. 4. In level three, our text is exposed a typical type I and analyzed Inquiry Index 15-25 ( Middle ), but FAST is found type IV, excepting Manipulating Apparatus and Observation and analyzed Inquiry Index over 35 ( Very - High ). Therefore, FAST Program is proved to be the desirable Inquiry Science Curricular Materials. In future, this worker is to arrange the results of the following paper as follows ; 1. The verification of the FAST Program by means of the Integrated Science Curricular Materials. 2. The development of the Inquiring, Integrated Science Curricular Materials through the results of the preceding study.

  • PDF

Machine Printed and Handwritten Text Discrimination in Korean Document Images

  • Trieu, Son Tung;Lee, Guee Sang
    • Smart Media Journal
    • /
    • v.5 no.3
    • /
    • pp.30-34
    • /
    • 2016
  • Nowadays, there are a lot of Korean documents, which often need to be identified in one of printed or handwritten text. Early methods for the identification use structural features, which can be simple and easy to apply to text of a specific font, but its performance depends on the font type and characteristics of the text. Recently, the bag-of-words model has been used for the identification, which can be invariant to changes in font size, distortions or modifications to the text. The method based on bag-of-words model includes three steps: word segmentation using connected component grouping, feature extraction, and finally classification using SVM(Support Vector Machine). In this paper, bag-of-words model based method is proposed using SURF(Speeded Up Robust Feature) for the identification of machine printed and handwritten text in Korean documents. The experiment shows that the proposed method outperforms methods based on structural features.

Text Mining of Wood Science Research Published in Korean and Japanese Journals

  • Eun-Suk JANG
    • Journal of the Korean Wood Science and Technology
    • /
    • v.51 no.6
    • /
    • pp.458-469
    • /
    • 2023
  • Text mining techniques provide valuable insights into research information across various fields. In this study, text mining was used to identify research trends in wood science from 2012 to 2022, with a focus on representative journals published in Korea and Japan. Abstracts from Journal of the Korean Wood Science and Technology (JKWST, 785 articles) and Journal of Wood Science (JWS, 812 articles) obtained from the SCOPUS database were analyzed in terms of the word frequency (specifically, term frequency-inverse document frequency) and co-occurrence network analysis. Both journals showed a significant occurrence of words related to the physical and mechanical properties of wood. Furthermore, words related to wood species native to each country and their respective timber industries frequently appeared in both journals. CLT was a common keyword in engineering wood materials in Korea and Japan. In addition, the keywords "MDF," "MUF," and "GFRP" were ranked in the top 50 in Korea. Research on wood anatomy was inferred to be more active in Japan than in Korea. Co-occurrence network analysis showed that words related to the physical and structural characteristics of wood were organically related to wood materials.

A Quantitative Approach to a Similarity Analysis on the Culinary Manuscripts in the Chosun Periods (계량적 접근에 의한 조선시대 필사본 조리서의 유사성 분석)

  • Lee, Ki-Hwang;Lee, Jae-Yun;Paek, Doo-Hyun
    • Language and Information
    • /
    • v.14 no.2
    • /
    • pp.131-157
    • /
    • 2010
  • This article reports an attempt to perform a similarity analysis on a collection of 25 culinary manuscripts in Chosun periods using a set of quantitative text analysis methods. Historical culinary texts are valuable resources for linguistic, historic, and cultural studies. We consider the similarity of two texts as the distributional similarities of the functional components of the texts. In the case of culinary texts, text elements such as food names, cooking methods, and ingredients are regarded as functional components. We derive the similarity information from the distributional characteristics of the two key functional components, cooking methods and ingredients. The results are also quantified and visualized to achieve a better understanding of the properties of the individual texts and the collection of the texts as a whole.

  • PDF

A Comparative Study of Textuality in Korean-Thai Female Poems -Feminism Point of View- (한·태 여성시의 텍스트성 비교 -페미니즘적 관점에서-)

  • Lim, Myung Sook
    • The Southeast Asian review
    • /
    • v.21 no.2
    • /
    • pp.263-291
    • /
    • 2011
  • The purpose of this study is to see and compare the contemporary Korean-Thai female poems from a feminine standpoint to newly clarify the textuality of their poems. The textuality defined in this manuscript is the text of Korean-Thai contemporary female poems. To newly clarify the textuality of their poems are to go against the existing discussion method and to newly read out the text as re-vision method. This discussion is to analyse deeply how the central exis composing a text which is the identity of woman in a body, appearance of uttrance, or action of abjection is exposed in gender space and to identify the poem's textuality. In other words, through in-depth analysis of the text of poems, which are very complicated as a skein of yarn, place a high value of Korean-Thai female poems. Transcending time, nations and races, if the text of female poem would not free from a biased male-dominated thinking or make a mystery of female poem textuality without critics or tend to be stereotype the text of poem as pathos of female, it would not get out from man-centered reading. To escape from the state of sexual discrimination, the new reading method was seriously analysed and found out that the female text poems not only implicate sexual discrimination but also link to expansive cultural and social structure. And for that reason, this study raise a question to male-dominant sexual discriminated norm. It is very significant that through this elaborate and in-depth text poem analysis, a creation process of female poem is traced. Eventually, the comparative study on Korean-Thai female poems is meaningful and worthy in regard to the contribution to promotion of cultural exchange between korea-Thai two nations and furthermore extend to East Asia to make a basement for the vitalization of Asia comparative literature.