• 제목/요약/키워드: Text Mining Method

Search Result 448, Processing Time 0.024 seconds

Clickstream Big Data Mining for Demographics based Digital Marketing (인구통계특성 기반 디지털 마케팅을 위한 클릭스트림 빅데이터 마이닝)

  • Park, Jiae;Cho, Yoonho
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.3
    • /
    • pp.143-163
    • /
    • 2016
  • The demographics of Internet users are the most basic and important sources for target marketing or personalized advertisements on the digital marketing channels which include email, mobile, and social media. However, it gradually has become difficult to collect the demographics of Internet users because their activities are anonymous in many cases. Although the marketing department is able to get the demographics using online or offline surveys, these approaches are very expensive, long processes, and likely to include false statements. Clickstream data is the recording an Internet user leaves behind while visiting websites. As the user clicks anywhere in the webpage, the activity is logged in semi-structured website log files. Such data allows us to see what pages users visited, how long they stayed there, how often they visited, when they usually visited, which site they prefer, what keywords they used to find the site, whether they purchased any, and so forth. For such a reason, some researchers tried to guess the demographics of Internet users by using their clickstream data. They derived various independent variables likely to be correlated to the demographics. The variables include search keyword, frequency and intensity for time, day and month, variety of websites visited, text information for web pages visited, etc. The demographic attributes to predict are also diverse according to the paper, and cover gender, age, job, location, income, education, marital status, presence of children. A variety of data mining methods, such as LSA, SVM, decision tree, neural network, logistic regression, and k-nearest neighbors, were used for prediction model building. However, this research has not yet identified which data mining method is appropriate to predict each demographic variable. Moreover, it is required to review independent variables studied so far and combine them as needed, and evaluate them for building the best prediction model. The objective of this study is to choose clickstream attributes mostly likely to be correlated to the demographics from the results of previous research, and then to identify which data mining method is fitting to predict each demographic attribute. Among the demographic attributes, this paper focus on predicting gender, age, marital status, residence, and job. And from the results of previous research, 64 clickstream attributes are applied to predict the demographic attributes. The overall process of predictive model building is compose of 4 steps. In the first step, we create user profiles which include 64 clickstream attributes and 5 demographic attributes. The second step performs the dimension reduction of clickstream variables to solve the curse of dimensionality and overfitting problem. We utilize three approaches which are based on decision tree, PCA, and cluster analysis. We build alternative predictive models for each demographic variable in the third step. SVM, neural network, and logistic regression are used for modeling. The last step evaluates the alternative models in view of model accuracy and selects the best model. For the experiments, we used clickstream data which represents 5 demographics and 16,962,705 online activities for 5,000 Internet users. IBM SPSS Modeler 17.0 was used for our prediction process, and the 5-fold cross validation was conducted to enhance the reliability of our experiments. As the experimental results, we can verify that there are a specific data mining method well-suited for each demographic variable. For example, age prediction is best performed when using the decision tree based dimension reduction and neural network whereas the prediction of gender and marital status is the most accurate by applying SVM without dimension reduction. We conclude that the online behaviors of the Internet users, captured from the clickstream data analysis, could be well used to predict their demographics, thereby being utilized to the digital marketing.

Unstructured Data Analysis using Equipment Check Ledger: A Case Study in Telecom Domain (장비점검 일지의 비정형 데이터분석을 통한 고장 대응 효율화 사례 연구)

  • Ju, Yeonjin;Kim, Yoosin;Jeong, Seung Ryul
    • Journal of Internet Computing and Services
    • /
    • v.21 no.1
    • /
    • pp.127-135
    • /
    • 2020
  • As the importance of the use and analysis of big data is emerging, there is a growing interest in natural language processing techniques for unstructured data such as news articles and comments. Particularly, as the collection of big data becomes possible, data mining techniques capable of pre-processing and analyzing data are emerging. In this case study with a telecom company, we propose a methodology how to formalize unstructured data using text mining. The domain is determined as equipment failure and the data is about 2.2 million equipment check ledger data. Data on equipment failures by 800,000 per year is accumulated in the equipment check ledger. The equipment check ledger coexist with both formal and unstructured data. Although formal data can be easily used for analysis, unstructured data is difficult to be used immediately for analysis. However, in unstructured data, there is a high possibility that important information. Because it can be contained that is not written in a formal. Therefore, in this study, we study to develop digital transformation method for unstructured data in equipment check ledger.

The Analysis of Public Awareness about Literary Therapy by Utilizing Big Data Analysis - The aspects of convergence literature and statistics (빅데이터 분석을 통한 문학치료의 대중적 인지도 분석 - 국문학과 통계학의 융합적 측면)

  • Choi, Kyoung-Ho;Park, Jeong-Hye
    • Journal of Digital Convergence
    • /
    • v.13 no.4
    • /
    • pp.395-404
    • /
    • 2015
  • This study is exploring objective awareness of literary therapy by consideration of popular perception about literary therapy through analysis of big data. The purpose of this study is the deduction of meaning information through analysis in the viewpoint of big data at online social network service(SNS) about 'literary therapy'. Accordingly, the main way of research became content analysis of keyword linked to literary therapy by utilizing opinion mining method related to text mining. The study mainly grasped 'literary therapy' and analyzed 'bibliotherapy' comparatively. The period of study was from Oct. 10th to Nov. 10th, 2014(during 30 days), and SNS such as blog or twitter became the subject of search. Through the result of study analysis, the conclusion that the spread of literary therapeutic prospect, structural harmony of literary therapeutic field, and the solidity of perceptional axis about literary therapy are needed can be drawn. This study is worthwhile because it can investigate popular awareness about literary therapy and can suggest alternative for invigoration of literary therapy.

A Study on the Value Factors of Culture Consumers for Corporate Culture Marketing through Big Data Techniques (빅데이터 기법을 통한 기업 문화마케팅을 위한 문화소비자의 가치 요소 연구)

  • Oh, Se Jong
    • The Journal of the Convergence on Culture Technology
    • /
    • v.6 no.1
    • /
    • pp.31-36
    • /
    • 2020
  • Corporate Culture Marketing is a marketing tool that enhances a company's cultural image or conveys its image through culture. Culture Consumer value analysis is important predictive data in identifying the value and pursuit of life in individual consumption behavior, explaining the choice behavior of culture consumers, and serves as the basis for decision making. The research method was linked to the text mining and opinion mining techniques of big data, and extracted positive, negative and neutral words. The analysis targets culture consumers participating in concerts at Hyundai Card's 'Super Concert', which is subject to domestic consumers, and CJ ENM's 'KCON', which is subject to foreign consumers. The culture consumer value elements of corporate culture marketing are the basic conditions, and they were derived as 'Consensus Communication (Expression of Sensibility)', 'Participation Sharing(VIP Belonging)', 'Social Change Issue', 'Differentiating Services', 'Price Discount Benefit' and 'Location Quality'. In the future, we will need to foster 'Culture Technology Marketers' and apply them in areas such as arts management planning, cultural investment, cultural distribution, cultural space, Corporate Culture, CSR and K-pop marketing to enhance corporate interests and brand value and enhance brand value.

Predicting Functional Outcomes of Patients With Stroke Using Machine Learning: A Systematic Review (머신러닝을 활용한 뇌졸중 환자의 기능적 결과 예측: 체계적 고찰)

  • Bae, Suyeong;Lee, Mi Jung;Nam, Sanghun;Hong, Ickpyo
    • Therapeutic Science for Rehabilitation
    • /
    • v.11 no.4
    • /
    • pp.23-39
    • /
    • 2022
  • Objective : To summarize clinical and demographic variables and machine learning uses for predicting functional outcomes of patients with stroke. Methods : We searched PubMed, CINAHL and Web of Science to identify published articles from 2010 to 2021. The search terms were "machine learning OR data mining AND stroke AND function OR prediction OR/AND rehabilitation". Articles exclusively using brain imaging techniques, deep learning method and articles without available full text were excluded in this study. Results : Nine articles were selected for this study. Support vector machines (19.05%) and random forests (19.05%) were two most frequently used machine learning models. Five articles (55.56%) demonstrated that the impact of patient initial and/or discharge assessment scores such as modified ranking scale (mRS) or functional independence measure (FIM) on stroke patients' functional outcomes was higher than their clinical characteristics. Conclusions : This study showed that patient initial and/or discharge assessment scores such as mRS or FIM could influence their functional outcomes more than their clinical characteristics. Evaluating and reviewing initial and or discharge functional outcomes of patients with stroke might be required to develop the optimal therapeutic interventions to enhance functional outcomes of patients with stroke.

How to improve the accuracy of recommendation systems: Combining ratings and review texts sentiment scores (평점과 리뷰 텍스트 감성분석을 결합한 추천시스템 향상 방안 연구)

  • Hyun, Jiyeon;Ryu, Sangyi;Lee, Sang-Yong Tom
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.1
    • /
    • pp.219-239
    • /
    • 2019
  • As the importance of providing customized services to individuals becomes important, researches on personalized recommendation systems are constantly being carried out. Collaborative filtering is one of the most popular systems in academia and industry. However, there exists limitation in a sense that recommendations were mostly based on quantitative information such as users' ratings, which made the accuracy be lowered. To solve these problems, many studies have been actively attempted to improve the performance of the recommendation system by using other information besides the quantitative information. Good examples are the usages of the sentiment analysis on customer review text data. Nevertheless, the existing research has not directly combined the results of the sentiment analysis and quantitative rating scores in the recommendation system. Therefore, this study aims to reflect the sentiments shown in the reviews into the rating scores. In other words, we propose a new algorithm that can directly convert the user 's own review into the empirically quantitative information and reflect it directly to the recommendation system. To do this, we needed to quantify users' reviews, which were originally qualitative information. In this study, sentiment score was calculated through sentiment analysis technique of text mining. The data was targeted for movie review. Based on the data, a domain specific sentiment dictionary is constructed for the movie reviews. Regression analysis was used as a method to construct sentiment dictionary. Each positive / negative dictionary was constructed using Lasso regression, Ridge regression, and ElasticNet methods. Based on this constructed sentiment dictionary, the accuracy was verified through confusion matrix. The accuracy of the Lasso based dictionary was 70%, the accuracy of the Ridge based dictionary was 79%, and that of the ElasticNet (${\alpha}=0.3$) was 83%. Therefore, in this study, the sentiment score of the review is calculated based on the dictionary of the ElasticNet method. It was combined with a rating to create a new rating. In this paper, we show that the collaborative filtering that reflects sentiment scores of user review is superior to the traditional method that only considers the existing rating. In order to show that the proposed algorithm is based on memory-based user collaboration filtering, item-based collaborative filtering and model based matrix factorization SVD, and SVD ++. Based on the above algorithm, the mean absolute error (MAE) and the root mean square error (RMSE) are calculated to evaluate the recommendation system with a score that combines sentiment scores with a system that only considers scores. When the evaluation index was MAE, it was improved by 0.059 for UBCF, 0.0862 for IBCF, 0.1012 for SVD and 0.188 for SVD ++. When the evaluation index is RMSE, UBCF is 0.0431, IBCF is 0.0882, SVD is 0.1103, and SVD ++ is 0.1756. As a result, it can be seen that the prediction performance of the evaluation point reflecting the sentiment score proposed in this paper is superior to that of the conventional evaluation method. In other words, in this paper, it is confirmed that the collaborative filtering that reflects the sentiment score of the user review shows superior accuracy as compared with the conventional type of collaborative filtering that only considers the quantitative score. We then attempted paired t-test validation to ensure that the proposed model was a better approach and concluded that the proposed model is better. In this study, to overcome limitations of previous researches that judge user's sentiment only by quantitative rating score, the review was numerically calculated and a user's opinion was more refined and considered into the recommendation system to improve the accuracy. The findings of this study have managerial implications to recommendation system developers who need to consider both quantitative information and qualitative information it is expect. The way of constructing the combined system in this paper might be directly used by the developers.

Categorizing Sub-Categories of Mobile Application Services using Network Analysis: A Case of Healthcare Applications (네트워크 분석을 이용한 애플리케이션 서비스 하위 카테고리 분류: 헬스케어 어플리케이션 중심으로)

  • Ha, Sohee;Geum, Youngjung
    • The Journal of Society for e-Business Studies
    • /
    • v.25 no.3
    • /
    • pp.15-40
    • /
    • 2020
  • Due to the explosive growth of mobile application services, categorizing mobile application services is in need in practice from both customers' and developers' perspectives. Despite the fact, however, there have been limited studies regarding systematic categorization of mobile application services. In response, this study proposed a method for categorizing mobile application services, and suggested a service taxonomy based on the network clustering results. Total of 1,607 mobile healthcare services are collected through the Google Play store. The network analysis is conducted based on the similarity of descriptions in each application service. Modularity detection analysis is conducted to detects communities in the network, and service taxonomy is derived based on each cluster. This study is expected to provide a systematic approach to the service categorization, which is helpful to both customers who want to navigate mobile application service in a systematic manner and developers who desire to analyze the trend of mobile application services.

AI Technology Analysis using Partial Least Square Regression

  • Choi, JunHyeog;Jun, Sunghae
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.3
    • /
    • pp.109-115
    • /
    • 2020
  • In this paper, we propose an artificial intelligence(AI) technology analysis using partial least square(PLS) regression model. AI technology is now affecting most areas of our society. So, it is necessary to understand this technology. To analyze the AI technology, we collect the patent documents related to AI from the patent databases in the world. We extract AI technology keywords from the patent documents by text mining techniques. In addition, we analyze the AI keyword data by PLS regression model. This regression model is based on the technique of partial least squares used in the advanced analyses such as bioinformatics, social science, and engineering. To show the performance of our proposed method, we make experiments using AI patent documents, and we illustrate how our research can be applied to real problems. This paper is applicable not only to AI technology but also to other technological fields. This also contributes to understanding other various technologies by PLS regression analysis.

A Study on Collecting and Structuring Language Resource for Named Entity Recognition and Relation Extraction from Biomedical Abstracts (생의학 분야 학술 논문에서의 개체명 인식 및 관계 추출을 위한 언어 자원 수집 및 통합적 구조화 방안 연구)

  • Kang, Seul-Ki;Choi, Yun-Soo;Choi, Sung-Pil
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.51 no.4
    • /
    • pp.227-248
    • /
    • 2017
  • This paper introduces an integrated model for systematically constructing a linguistic resource database that can be used by machine learning-based biomedical information extraction systems. The proposed method suggests an orderly process of collecting and constructing dictionaries and training sets for both named-entity recognition and relation extraction. Multiple heterogeneous structures for the resources which are collected from diverse sources are analyzed to derive essential items and fields for constructing the integrated database. All the collected resources are converted and refined to build an integrated linguistic resource storage. In this paper, we constructed entity dictionaries of gene, protein, disease and drug, which are considered core linguistic elements or core named entities in the biomedical domains and conducted verification tests to measure their acceptability.

Extracting Method of User's Interests by Using SNS Follower's Relationship and Sequential Pattern Evaluation Indices for Keyword (키워드를 위한 시퀀셜 패턴 평가 지표와 SNS 팔로워의 관계를 이용한 사용자 관심사항 추출방법)

  • Shin, Bong-Hi;Jeon, Hye-Kyoung
    • Journal of the Korea Convergence Society
    • /
    • v.8 no.8
    • /
    • pp.71-75
    • /
    • 2017
  • Due to the spread of SNS, web-based consumer-generated data is increasing exponentially. It is important in many fields to accurately extract what is appropriate for the user's interest in a large amount of data. It is especially important for business mangers to establish marketing policies to find the right customers for them in many users. In this paper, we try to obtain important information centering on customers who are interested in each account through Twitter follow - following relationship. Because Twitter's current follower relationships do not reflect the user's interests, we try to figure out the details of interest using keyword extraction methods for tweets of followers. To do this, we select two domestic commercial Twitter accounts and apply the sequential pattern evaluation index to the mining key phrase of the text data collected from the follower.