• 제목/요약/키워드: Text Mining Method

검색결과 448건 처리시간 0.021초

Design and Implementation of Incremental Learning Technology for Big Data Mining

  • Min, Byung-Won;Oh, Yong-Sun
    • International Journal of Contents
    • /
    • 제15권3호
    • /
    • pp.32-38
    • /
    • 2019
  • We usually suffer from difficulties in treating or managing Big Data generated from various digital media and/or sensors using traditional mining techniques. Additionally, there are many problems relative to the lack of memory and the burden of the learning curve, etc. in an increasing capacity of large volumes of text when new data are continuously accumulated because we ineffectively analyze total data including data previously analyzed and collected. In this paper, we propose a general-purpose classifier and its structure to solve these problems. We depart from the current feature-reduction methods and introduce a new scheme that only adopts changed elements when new features are partially accumulated in this free-style learning environment. The incremental learning module built from a gradually progressive formation learns only changed parts of data without any re-processing of current accumulations while traditional methods re-learn total data for every adding or changing of data. Additionally, users can freely merge new data with previous data throughout the resource management procedure whenever re-learning is needed. At the end of this paper, we confirm a good performance of this method in data processing based on the Big Data environment throughout an analysis because of its learning efficiency. Also, comparing this algorithm with those of NB and SVM, we can achieve an accuracy of approximately 95% in all three models. We expect that our method will be a viable substitute for high performance and accuracy relative to large computing systems for Big Data analysis using a PC cluster environment.

A review of big data analytics and healthcare (빅데이터 분석과 헬스케어에 대한 동향)

  • Moon, Seok-Jae;Lee, Namju
    • Journal of the Korean Applied Science and Technology
    • /
    • 제37권1호
    • /
    • pp.76-82
    • /
    • 2020
  • Big data analysis in healthcare research seems to be a necessary strategy for the convergence of sports science and technology in the era of the Fourth Industrial Revolution. The purpose of this study is to provide the basic review to secure the diversity of big data and healthcare convergence by discussing the concept, analysis method, and application examples of big data and by exploring the application. Text mining, data mining, opinion mining, process mining, cluster analysis, and social network analysis is currently used. Identifying high-risk factor for a certain condition, determining specific health determinants for diseases, monitoring bio signals, predicting diseases, providing training and treatments, and analyzing healthcare measurements would be possible via big data analysis. As a further work, the big data characteristics provide very appropriate basis to use promising software platforms for development of applications that can handle big data in healthcare and even more in sports science.

Multidimensional Analysis of Consumers' Opinions from Online Product Reviews

  • Taewook Kim;Dong Sung Kim;Donghyun Kim;Jong Woo Kim
    • Asia pacific journal of information systems
    • /
    • 제29권4호
    • /
    • pp.838-855
    • /
    • 2019
  • Online product reviews are a vital source for companies in that they contain consumers' opinions of products. The earlier methods of opinion mining, which involve drawing semantic information from text, have been mostly applied in one dimension. This is not sufficient in itself to elicit reviewers' comprehensive views on products. In this paper, we propose a novel approach in opinion mining by projecting online consumers' reviews in a multidimensional framework to improve review interpretation of products. First of all, we set up a new framework consisting of six dimensions based on a marketing management theory. To calculate the distances of review sentences and each dimension, we embed words in reviews utilizing Google's pre-trained word2vector model. We classified each sentence of the reviews into the respective dimensions of our new framework. After the classification, we measured the sentiment degrees for each sentence. The results were plotted using a radar graph in which the axes are the dimensions of the framework. We tested the strategy on Amazon product reviews of the iPhone and Galaxy smartphone series with a total of around 21,000 sentences. The results showed that the radar graphs visually reflected several issues associated with the products. The proposed method is not for specific product categories. It can be generally applied for opinion mining on reviews of any product category.

An Experimental Study on Selecting Association Terms Using Text Mining Techniques (텍스트 마이닝 기법을 이용한 연관용어 선정에 관한 실험적 연구)

  • Kim, Su-Yeon;Chung, Young-Mee
    • Journal of the Korean Society for information Management
    • /
    • 제23권3호
    • /
    • pp.147-165
    • /
    • 2006
  • In this study, experiments for selection of association terms were conducted in order to discover the optimum method in selecting additional terms that are related to an initial query term. Association term sets were generated by using support, confidence, and lift measures of the Apriori algorithm, and also by using the similarity measures such as GSS, Jaccard coefficient, cosine coefficient, and Sokal & Sneath 5, and mutual information. In performance evaluation of term selection methods, precision of association terms as well as the overlap ratio of association terms and relevant documents' indexing terms were used. It was found that Apriori algorithm and GSS achieved the highest level of performances.

Measuring the Confidence of Human Disaster Risk Case based on Text Mining (텍스트마이닝 기반의 인적재난사고사례 신뢰도 측정연구)

  • Lee, Young-Jai;Lee, Sung-Soo
    • The Journal of Information Systems
    • /
    • 제20권3호
    • /
    • pp.63-79
    • /
    • 2011
  • Deducting the risk level of infrastructure and buildings based on past human disaster risk cases and implementing prevention measures are important activities for disaster prevention. The object of this study is to measure the confidence to proceed quantitative analysis of various disaster risk cases through text mining methodology. Indeed, by examining confidence calculation process and method, this study suggests also a basic quantitative framework. The framework to measure the confidence is composed into four stages. First step describes correlation by categorizing basic elements based on human disaster ontology. Secondly, terms and cases of Term-Document Matrix will be created and the frequency of certain cases and terms will be quantified, the correlation value will be added to the missing values. In the third stage, association rules will be created according to the basic elements of human disaster risk cases. Lastly, the confidence value of disaster risk cases will be measured through association rules. This kind of confidence value will become a key element when deciding a risk level of a new disaster risk, followed up by preventive measures. Through collection of human disaster risk cases related to road infrastructure, this study will demonstrate a case where the four steps of the quantitative framework and process had been actually used for verification.

Design of a Sentiment Analysis System to Prevent School Violence and Student's Suicide (학교폭력과 자살사고를 예방하기 위한 감성분석 시스템의 설계)

  • Kim, YoungTaek
    • The Journal of Korean Association of Computer Education
    • /
    • 제17권6호
    • /
    • pp.115-122
    • /
    • 2014
  • One of the problems with current youth generations is increasing rate of violence and suicide in their school lives, and this study aims at the design of a sentiment analysis system to prevent suicide by uising big data process. The main issues of the design are economical implementation, easy and fast processing for the users, so, the open source Hadoop system with MapReduce algorithm is used on the HDFS(Hadoop Distributed File System) for the experimentation. This study uses word count method to do the sentiment analysis with informal data on some sns communications concerning a kinds of violent words, in terms of text mining to avoid some expensive and complex statistical analysis methods.

  • PDF

Patent Keyword Analysis for Forecasting Emerging Technology : GHG Technology (부상기술 예측을 위한 특허키워드정보분석에 관한 연구 - GHG 기술 중심으로)

  • Choe, Do Han;Kim, Gab Jo;Park, Sang Sung;Jang, Dong Sik
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • 제9권2호
    • /
    • pp.139-149
    • /
    • 2013
  • As the importance of technology forecasting while countries and companies manage the R&D project is growing bigger, the methodology of technology forecasting has been diversified. One of the forecasting method is patent analysis. This research proposes quick forecasting process of emerging technology based on keyword approach using text mining. The forecasting process is following: First, the term-document matrix is extracted from patent documents by using text mining. Second, emerging technology keyword are extracted by analyzing the importance of word from utilizing mean values and standard deviation values of the term and the emerging trend of word discovered from time series information of the term. Next, association between terms is measured by using cosine similarity. finally, the keyword of emerging technology is selected in consequence of the synthesized result and we forecast the emerging technology according to the results. The technology forecasting process described in this paper can be applied to developing computerized technology forecasting system integrated with various results of other patent analysis for decision maker of company and country.

Analysis on Topic Trends and Topic Modeling of KSHSM Journal Papers using Text Mining (텍스트마이닝을 활용한 보건의료산업학회지의 토픽 모델링 및 토픽트렌드 분석)

  • Cho, Kyoung-Won;Bae, Sung-Kwon;Woo, Young-Woon
    • The Korean Journal of Health Service Management
    • /
    • 제11권4호
    • /
    • pp.213-224
    • /
    • 2017
  • Objectives : The purpose of this study was to analyze representative topics and topic trends of papers in Korean Society and Health Service Management(KSHSM) Journal. Methods : We collected English abstracts and key words of 516 papers in KSHSM Journal from 2007 to 2017. We utilized Python web scraping programs for collecting the papers from Korea Citation Index web site, and RStudio software for topic analysis based on latent Dirichlet allocation algorithm. Results : 9 topics were decided as the best number of topics by perplexity analysis and the resultant 9 topics for all the papers were extracted using Gibbs sampling method. We could refine 9 topics to 5 topics by deep consideration of meanings of each topics and analysis of intertopic distance map. In topic trends analysis from 2007 to 2017, we could verify 'Health Management' and 'Hospital Service' were two representative topics, and 'Hospital Service' was prevalent topic by 2011, but the ratio of the two topics became to be similar from 2012. Conclusions : We discovered 5 topics were the best number of topics and the topic trends reflected the main issues of KSHSM Journal, such as name revision of the society in 2012.

The Strategy of Wireless Power Transfer for Light Rail Transit By Core Technologies Analysis Based on Text Mining

  • Meng, Xiang-Yu;Han, Young-Jae;Eum, Soo-Min;Cho, Sung-Won
    • Journal of the Korea Society of Computer and Information
    • /
    • 제23권11호
    • /
    • pp.193-201
    • /
    • 2018
  • In this paper, we extracted relevant patent data and conducted statistical analysis to understand the technical development trend related to Wireless Power Transfer (WPT) for Light Rail Transit (LRT). Recently, with the development of WPT technologies, the Light Rail Transit (LRT) industry is concentrating on applying WPT to the power supply system of trains because of their advantages compared wired counterpart, such as low maintenance cost and high stability. This technology is divided into three areas: wireless feeding and collecting technology, high-frequency power converter technology and orbital and infrastructure technology. From each specific area, key words in patent document were extracted by TF-IDF method and analyzed by social network. In the keyword network, core word of each specific technology were extracted according to their degree centrality. Then, the multi-word phrases were also built to represent the concept of core technologies. Finally, based on the analysis results, the development strategies for each specifics technical area of WPT in LRT filed will be provided.

A Study on Effective Sentiment Analysis through News Classification in Bankruptcy Prediction Model (부도예측 모형에서 뉴스 분류를 통한 효과적인 감성분석에 관한 연구)

  • Kim, Chansong;Shin, Minsoo
    • Journal of Information Technology Services
    • /
    • 제18권1호
    • /
    • pp.187-200
    • /
    • 2019
  • Bankruptcy prediction model is an issue that has consistently interested in various fields. Recently, as technology for dealing with unstructured data has been developed, researches applied to business model prediction through text mining have been activated, and studies using this method are also increasing in bankruptcy prediction. Especially, it is actively trying to improve bankruptcy prediction by analyzing news data dealing with the external environment of the corporation. However, there has been a lack of study on which news is effective in bankruptcy prediction in real-time mass-produced news. The purpose of this study was to evaluate the high impact news on bankruptcy prediction. Therefore, we classify news according to type, collection period, and analyzed the impact on bankruptcy prediction based on sentiment analysis. As a result, artificial neural network was most effective among the algorithms used, and commentary news type was most effective in bankruptcy prediction. Column and straight type news were also significant, but photo type news was not significant. In the news by collection period, news for 4 months before the bankruptcy was most effective in bankruptcy prediction. In this study, we propose a news classification methods for sentiment analysis that is effective for bankruptcy prediction model.