• Title/Summary/Keyword: 텍스트 연구

Search Result 3,492, Processing Time 0.032 seconds

Feature selection for text data via sparse principal component analysis (희소주성분분석을 이용한 텍스트데이터의 단어선택)

  • Won Son
    • The Korean Journal of Applied Statistics
    • /
    • v.36 no.6
    • /
    • pp.501-514
    • /
    • 2023
  • When analyzing high dimensional data such as text data, if we input all the variables as explanatory variables, statistical learning procedures may suffer from over-fitting problems. Furthermore, computational efficiency can deteriorate with a large number of variables. Dimensionality reduction techniques such as feature selection or feature extraction are useful for dealing with these problems. The sparse principal component analysis (SPCA) is one of the regularized least squares methods which employs an elastic net-type objective function. The SPCA can be used to remove insignificant principal components and identify important variables from noisy observations. In this study, we propose a dimension reduction procedure for text data based on the SPCA. Applying the proposed procedure to real data, we find that the reduced feature set maintains sufficient information in text data while the size of the feature set is reduced by removing redundant variables. As a result, the proposed procedure can improve classification accuracy and computational efficiency, especially for some classifiers such as the k-nearest neighbors algorithm.

A Study on the Effect of Using Sentiment Lexicon in Opinion Classification (오피니언 분류의 감성사전 활용효과에 대한 연구)

  • Kim, Seungwoo;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.1
    • /
    • pp.133-148
    • /
    • 2014
  • Recently, with the advent of various information channels, the number of has continued to grow. The main cause of this phenomenon can be found in the significant increase of unstructured data, as the use of smart devices enables users to create data in the form of text, audio, images, and video. In various types of unstructured data, the user's opinion and a variety of information is clearly expressed in text data such as news, reports, papers, and various articles. Thus, active attempts have been made to create new value by analyzing these texts. The representative techniques used in text analysis are text mining and opinion mining. These share certain important characteristics; for example, they not only use text documents as input data, but also use many natural language processing techniques such as filtering and parsing. Therefore, opinion mining is usually recognized as a sub-concept of text mining, or, in many cases, the two terms are used interchangeably in the literature. Suppose that the purpose of a certain classification analysis is to predict a positive or negative opinion contained in some documents. If we focus on the classification process, the analysis can be regarded as a traditional text mining case. However, if we observe that the target of the analysis is a positive or negative opinion, the analysis can be regarded as a typical example of opinion mining. In other words, two methods (i.e., text mining and opinion mining) are available for opinion classification. Thus, in order to distinguish between the two, a precise definition of each method is needed. In this paper, we found that it is very difficult to distinguish between the two methods clearly with respect to the purpose of analysis and the type of results. We conclude that the most definitive criterion to distinguish text mining from opinion mining is whether an analysis utilizes any kind of sentiment lexicon. We first established two prediction models, one based on opinion mining and the other on text mining. Next, we compared the main processes used by the two prediction models. Finally, we compared their prediction accuracy. We then analyzed 2,000 movie reviews. The results revealed that the prediction model based on opinion mining showed higher average prediction accuracy compared to the text mining model. Moreover, in the lift chart generated by the opinion mining based model, the prediction accuracy for the documents with strong certainty was higher than that for the documents with weak certainty. Most of all, opinion mining has a meaningful advantage in that it can reduce learning time dramatically, because a sentiment lexicon generated once can be reused in a similar application domain. Additionally, the classification results can be clearly explained by using a sentiment lexicon. This study has two limitations. First, the results of the experiments cannot be generalized, mainly because the experiment is limited to a small number of movie reviews. Additionally, various parameters in the parsing and filtering steps of the text mining may have affected the accuracy of the prediction models. However, this research contributes a performance and comparison of text mining analysis and opinion mining analysis for opinion classification. In future research, a more precise evaluation of the two methods should be made through intensive experiments.

A Study on Text Pattern Analysis Applying Discrete Fourier Transform - Focusing on Sentence Plagiarism Detection - (이산 푸리에 변환을 적용한 텍스트 패턴 분석에 관한 연구 - 표절 문장 탐색 중심으로 -)

  • Lee, Jung-Song;Park, Soon-Cheol
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.22 no.2
    • /
    • pp.43-52
    • /
    • 2017
  • Pattern Analysis is One of the Most Important Techniques in the Signal and Image Processing and Text Mining Fields. Discrete Fourier Transform (DFT) is Generally Used to Analyzing the Pattern of Signals and Images. We thought DFT could also be used on the Analysis of Text Patterns. In this Paper, DFT is Firstly Adapted in the World to the Sentence Plagiarism Detection Which Detects if Text Patterns of a Document Exist in Other Documents. We Signalize the Texts Converting Texts to ASCII Codes and Apply the Cross-Correlation Method to Detect the Simple Text Plagiarisms such as Cut-and-paste, term Relocations and etc. WordNet is using to find Similarities to Detect the Plagiarism that uses Synonyms, Translations, Summarizations and etc. The Data set, 2013 Corpus, Provided by PAN Which is the One of Well-known Workshops for Text Plagiarism is used in our Experiments. Our Method are Fourth Ranked Among the Eleven most Outstanding Plagiarism Detection Methods.

Text Structuring using Centering Theory (중심화 이론을 이용한 텍스트 구조화)

  • Roh, Ji-Eun;Na, Seung-Hoon;Lee, Jong-Hyeok
    • Journal of KIISE:Software and Applications
    • /
    • v.34 no.6
    • /
    • pp.572-583
    • /
    • 2007
  • This paper investigates Centering-based metrics to evaluate ordering of utterances for text structuring. We point out a problem of MIN.NOCB metric which has been regarded as the simplest and best measure to evaluate coherence of ordering within Centering framework, and propose a new Centering-based metric, MAX.CPS as an alternative or supplementary one. This paper introduces a framework which pre-estimates the effectiveness of a metric on a given input ordering, and selects an applicable metric according to the pre-estimation result. Using this framework, we propose a new policy which can generate more optimal ordering within Centering framework. Moreover, we evaluate several kinds of Cf-ranking methods in terms of Centering-based metrics, and find that simply ranking entities by their linear order is generally the most suitable because of characteristics in Korean.

Text Detection and Recognition in Outdoor Korean Signboards for Mobile System Applications (모바일 시스템 응용을 위한 실외 한국어 간판 영상에서 텍스트 검출 및 인식)

  • Park, J.H.;Lee, G.S.;Kim, S.H.;Lee, M.H.;Toan, N.D.
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.46 no.2
    • /
    • pp.44-51
    • /
    • 2009
  • Text understand in natural images has become an active research field in the past few decades. In this paper, we present an automatic recognition system in Korean signboards with a complex background. The proposed algorithm includes detection, binarization and extraction of text for the recognition of shop names. First, we utilize an elaborate detection algorithm to detect possible text region based on edge histogram of vertical and horizontal direction. And detected text region is segmented by clustering method. Second, the text is divided into individual characters based on connected components whose center of mass lie below the center line, which are recognized by using a minimum distance classifier. A shape-based statistical feature is adopted, which is adequate for Korean character recognition. The system has been implemented in a mobile phone and is demonstrated to show acceptable performance.

Design of Handwriting-based Text Interface for Support of Mobile Platform Education Contents (모바일 플랫폼 교육 콘텐츠 지원을 위한 손 글씨 기반 텍스트 인터페이스 설계)

  • Cho, Yunsik;Cho, Sae-Hong;Kim, Jinmo
    • Journal of the Korea Computer Graphics Society
    • /
    • v.27 no.5
    • /
    • pp.81-89
    • /
    • 2021
  • This study proposes a text interface for support of language-based educational contents in a mobile platform environment. The proposed interface utilizes deep learning as an input structure to write words through handwriting. Based on GUI (Graphical User Interface) using buttons and menus of mobile platform contents and input methods such as screen touch, click, and drag, we design a text interface that can directly input and process handwriting from the user. It uses the EMNIST (Extended Modified National Institute of Standards and Technology database) dataset and a trained CNN (Convolutional Neural Network) to classify and combine alphabetic texts to complete words. Finally, we conduct experiments to analyze the learning support effect of the interface proposed by directly producing English word education contents and to compare satisfaction. We compared the ability to learn English words presented by users who have experienced the existing keypad-type interface and the proposed handwriting-based text interface in the same educational environment, and we analyzed the overall satisfaction in the process of writing words by manipulating the interface.

HTML Text Extraction Using Tag Path and Text Appearance Frequency (태그 경로 및 텍스트 출현 빈도를 이용한 HTML 본문 추출)

  • Kim, Jin-Hwan;Kim, Eun-Gyung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.12
    • /
    • pp.1709-1715
    • /
    • 2021
  • In order to accurately extract the necessary text from the web page, the method of specifying the tag and style attributes where the main contents exist to the web crawler has a problem in that the logic for extracting the main contents. This method needs to be modified whenever the web page configuration is changed. In order to solve this problem, the method of extracting the text by analyzing the frequency of appearance of the text proposed in the previous study had a limitation in that the performance deviation was large depending on the collection channel of the web page. Therefore, in this paper, we proposed a method of extracting texts with high accuracy from various collection channels by analyzing not only the frequency of appearance of text but also parent tag paths of text nodes extracted from the DOM tree of web pages.

The Effects of Semantic Mapping as a Science Text Reading Strategy On High School Students' Inferential Comprehension (과학 텍스트 의미지도 읽기 전략이 고등학생의 추론적 이해에 미치는 영향)

  • Sujin Lee;Jihun Park;Jeonghee Nam
    • Journal of the Korean Chemical Society
    • /
    • v.67 no.5
    • /
    • pp.362-377
    • /
    • 2023
  • The purpose of this study was to investigate the effect of semantic mapping as a science text reading strategy on high school students' inferential understanding. For this purpose, eight science text reading classes were conducted a reading strategy using semantic mapping for 46 students in two science-focused classes in the third grade of a high school. To investigate the effects of semantic mapping reading strategy on students' inferential comprehension, students' pre- and post-reading ability tests results were analyzed. In order to find out the change in inferential comprehension, the level of the inferential comprehension was analyzed using the analysis framework for developed in this study. For the classification of inferential comprehension, the levels of the inferential comprehension were converted into scores. The results of the analysis of changes in students' inferential comprehension showed that semantic mapping reading strategy classes influenced the changes in high school students' inference, especially bridge inference and elaborative inference among sub-elements of inferential comprehension.

About the Post-Cinematic Characteristics and Desire Shown in a Film (영화 <파란만장>에 나타난 욕망과 포스트시네마적인 특성에 대하여)

  • Son, Seong-Woo
    • Journal of Korea Entertainment Industry Association
    • /
    • v.13 no.3
    • /
    • pp.121-129
    • /
    • 2019
  • This study aims to focus on the text analysis, production methods of text, and reproduction of production methods, based on a film (2010) taken by mobile devices. As a digital film in which the objects and images have no characteristics of index, this work has the post-cinematic attributes in the aspect of consumers' recipience. This thesis paid attention to the interactions between essential change and production/consumption throughout the whole film culture in the receptive aspect. Just as the main character is a mediator-shaman in the film, this film works as a mediating position of cinematic possibility. In this film, there are different kinds of mediation such as mediation of shaman inside the text, mediation of film in the relationship between text and consumers, and consumers' instrumental desire for others'tool outside the text. Outside the text, this relevant film stimulates the imitation desire of consumer subjects as others. In other words, this is connected to the desire of consumers who aim to create a digital film through mobile devices as an author. This is connected to Simondon's thinking in which such technical objects not only generate new relationships, but also become a revolutionary seed that newly collectivizes human society.

A Comparative Analysis of Success Factors Between Social Commerce and Multichannel Distribution Using Text Mining Techniques (텍스트마이닝 기법을 이용한 소셜커머스와 멀티채널 유통업체 간 성공요인 비교 연구)

  • Choi, Hyun-Seung;Kim, Ye-Sol;Cho, Hyuk-Jun;Kang, Ju-Young
    • The Journal of Bigdata
    • /
    • v.1 no.2
    • /
    • pp.35-44
    • /
    • 2016
  • Today there is a fierce competition between social commerce and multi-channel distribution in korea and it is need to do comparative analysis about success factors between social commerce and multi-channel distribution. Unlike the other studies that have only used survey method, this study analyzed the success factors between social commerce and multichannel distribution using text mining techniques. We expect that the result of the study not only gives the practical implication for making the competition strategy of the retailers but also contributes to the diverse extension research.

  • PDF