• Title/Summary/Keyword: Web Mining

Search Result 550, Processing Time 0.026 seconds

A Study on Extracting News Contents from News Web Pages (뉴스 웹 페이지에서 기사 본문 추출에 관한 연구)

  • Lee, Yong-Gu
    • Journal of the Korean Society for information Management
    • /
    • v.26 no.1
    • /
    • pp.305-320
    • /
    • 2009
  • The news pages provided through the web contain unnecessary information. This causes low performance and inefficiency of the news processing system. In this study, news content extraction methods, which are based on sentence identification and block-level tags news web pages, was suggested. To obtain optimal performance, combinations of these methods were applied. The results showed good performance when using an extraction method which applied the sentence identification and eliminated hyperlink text from web pages. Moreover, this method showed better results when combined with the extraction method which used block-level. Extraction methods, which used sentence identification, were effective for raising the extraction recall ratio.

The Evaluation for Web Mining and Analytics Service from the View of Personal Information Protection and Privacy (개인정보보호 관점에서의 웹 트래픽 수집 및 분석 서비스에 대한 타당성 연구)

  • Kang, Daniel;Shim, Mi-Na;Bang, Je-Wan;Lee, Sang-Jin;Lim, Jong-In
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.19 no.6
    • /
    • pp.121-134
    • /
    • 2009
  • Consumer-centric marketing business is surely one of the most successful emerging business but it poses a threat to personal privacy. Between the service provider and the user there are many contrary issues to each other. The enterprise asserts that to abuse the privacy data which is anonymous there is not a problem. The individual only will not be able to willingly submit the problem which is latent. Web traffic analysis technology itself doesn't create issues, but this technology when used on data of personal nature might cause concerns. The most criticized ethical issue involving web traffic analysis is the invasion of privacy. So we need to inspect how many and what kind of personal informations being used and if there is any illegal treatment of personal information. In this paper, we inspect the operation of consumer-centric marketing tools such as web log analysis solutions and data gathering services with web browser toolbar. Also we inspect Microsoft explorer-based toolbar application which records and analyzes personal web browsing pattern through reverse engineering technology. Finally, this identified and explored security and privacy requirement issues to develop more reliable solutions. This study is very important for the balanced development with personal privacy protection and web traffic analysis industry.

A Survey of Web Mining Focused on Web Structure Mining (웹 구조 마이닝에 초점을 둔 웹 마이닝의 조사)

  • Lee, Seok-Min;Park, Dae-Myeong;Yoo, Dae-Hun;Choi, Woong-Chul
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2010.06c
    • /
    • pp.334-338
    • /
    • 2010
  • 컴퓨터 기술의 발달 및 웹의 확산으로 인해 개인이 얻을 수 있는 정보의 양이 증가되었지만, 이로 인해 필요한 관련 정보를 탐색하는 것과 다량의 정보로부터 지식을 창출한다는 것이 어렵게 되었고, 고객 또는 사용자에 대한 학습 과정 및 정보의 개인화 등의 문제가 대두되게 되었다. 이러한 문제들을 해소하기 위해 웹으로부터 정보를 얻을 수 있는 자동화된 툴이 필요하게 되었고, 얻은 정보를 이용하여 웹 사용자들의 패턴을 식별할 수 있는 방법 또한 필요하게 되었다. 이러한 관심은 데이터 마이닝을 온라인에서 적용하고자 하는 노력으로 이어졌고, 현재 데이터 마이닝 기술을 온라인에 적용한 웹 마이닝 기술을 사용하고 있다. 웹 마이닝은 웹의 방대한 양의 자료 및 구조를 좀 더 유용하고, 효율적인 정보로 가공하여 사용자에게 제공할 수 있도록 도와주는 기술이다. 본 논문에서는 웹 마이닝의 전반적인 개념과 분류를 소개한다. 또한, 웹 마이닝의 분류 중 웹 구조 마이닝에 초점을 맞추어 개념 및 웹 구조 마이닝의 대표적인 알고리듬을 소개한다.

  • PDF

Analysis on Topic Trends and Topic Modeling of KSHSM Journal Papers using Text Mining (텍스트마이닝을 활용한 보건의료산업학회지의 토픽 모델링 및 토픽트렌드 분석)

  • Cho, Kyoung-Won;Bae, Sung-Kwon;Woo, Young-Woon
    • The Korean Journal of Health Service Management
    • /
    • v.11 no.4
    • /
    • pp.213-224
    • /
    • 2017
  • Objectives : The purpose of this study was to analyze representative topics and topic trends of papers in Korean Society and Health Service Management(KSHSM) Journal. Methods : We collected English abstracts and key words of 516 papers in KSHSM Journal from 2007 to 2017. We utilized Python web scraping programs for collecting the papers from Korea Citation Index web site, and RStudio software for topic analysis based on latent Dirichlet allocation algorithm. Results : 9 topics were decided as the best number of topics by perplexity analysis and the resultant 9 topics for all the papers were extracted using Gibbs sampling method. We could refine 9 topics to 5 topics by deep consideration of meanings of each topics and analysis of intertopic distance map. In topic trends analysis from 2007 to 2017, we could verify 'Health Management' and 'Hospital Service' were two representative topics, and 'Hospital Service' was prevalent topic by 2011, but the ratio of the two topics became to be similar from 2012. Conclusions : We discovered 5 topics were the best number of topics and the topic trends reflected the main issues of KSHSM Journal, such as name revision of the society in 2012.

How Facebook Functions in a Social Movement: An Examination Using the Web Mining Approach

  • Cao, Wenny;Cheong, Angus;Li, Zizi
    • Asian Journal for Public Opinion Research
    • /
    • v.1 no.4
    • /
    • pp.268-291
    • /
    • 2014
  • Social media is becoming more and more important in social movements. This study, adopting the web mining approach, attempts to investigate how social media, Facebook in particular, functioned in the "May 25 Protest" and the "May 27 Protest", two movements which broke out in Macao on 25 and 27 May 2014, respectively, against the Retirement Package Bill. In the two protests, Macao residents deployed Facebook to share information and motivated people's participation. Twelve events (181,106 people invited) and 36 groups/pages (41,266 members) related on Facebook were examined. Results showed that the information flow on Facebook fluctuated in accordance with the event development in reality. Multiple patterns of manifestation, such as video of adopted news or songs, designed profile (protest icon), original ironic pictures, self-organized clubs by undergraduates and white T-shirts as a symbol, among others, appeared online and interacted with offline actions. It was also found that social media assisted the information diffusion and provided persuasive reasons for netizens to join the movement. Social media helped to expand movement influence in providing a platform for diversified performances for actions taken in a protest, which could express and develop core and consistent movement repertoire.

Early Detection Assistance System for Rare Diseases based on Patient's Symptom Information (환자 증상정보 기반 희귀질환 조기 발견 보조시스템)

  • Jae-Min Choi;Sun-Yong Kim
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.18 no.2
    • /
    • pp.373-378
    • /
    • 2023
  • Untypical symptoms and lack of diagnostic records make it difficult for even medical specialists to detect rare diseases. Thus, it takes a lot of time and money from the onset of symptoms to an accurate diagnosis, which seriously results in physical, mental, and economic pressure on patients. In this paper, we propose and implement an early detection assistance system for rare diseases using web crawling and text mining, which can suggest the names of suspected rare diseases so that medical staffs can easily recall the disease names and make a final diagnosis of the rare diseases.

Design and Implementation of Web Server for Analyzing Clickstream (클릭스트림 분석을 위한 웹 서버 시스템의 설계 및 구현)

  • Kang, Mi-Jung;Jeong, Ok-Ran;Cho, Dong-Sub
    • The KIPS Transactions:PartD
    • /
    • v.9D no.5
    • /
    • pp.945-954
    • /
    • 2002
  • Clickstream is the information which demonstrate users' path through web sites. Analysis of clickstream shows how web sites are navigated and used by users. Clickstream of online web sites contains effective information of web marketing and to offers usefully personalized services to users, and helps us understand how users find web sites, what products they see, and what products they purchase. In this paper, we present an extended web log system that add to module of collection of clickstream to understand users' behavior patterns In web sites. This system offers the users clickstream information to database which can then analyze it with ease. Using ADO technology in store of database constructs extended web log server system. The process of making clickstreaming into database can facilitate analysis of various user patterns and generates aggregate profiles to offer personalized web service. In particular, our results indicate that by using the users' clickstream. We can achieve effective personalization of web sites.

Similarity Measurement with Interestingness Weight for Improving the Accuracy of Web Transaction Clustering (웹 트랜잭션 클러스터링의 정확성을 높이기 위한 흥미가중치 적용 유사도 비교방법)

  • Kang, Tae-Ho;Min, Young-Soo;Yoo, Jae-Soo
    • The KIPS Transactions:PartD
    • /
    • v.11D no.3
    • /
    • pp.717-730
    • /
    • 2004
  • Recently. many researches on the personalization of a web-site have been actively made. The web personalization predicts the sets of the most interesting URLs for each user through data mining approaches such as clustering techniques. Most existing methods using clustering techniques represented the web transactions as bit vectors that represent whether users visit a certain WRL or not to cluster web transactions. The similarity of the web transactions was decided according to the match degree of bit vectors. However, since the existing methods consider only whether users visit a certain URL or not, users' interestingness on the URL is excluded from clustering web transactions. That is, it is possible that the web transactions with different visit proposes or inclinations are classified into the same group. In this paper. we propose an enhanced transaction modeling with interestingness weight to solve such problems and a new similarity measuring method that exploits the proposed transaction modeling. It is shown through performance evaluation that our similarity measuring method improves the accuracy of the web transaction clustering over the existing method.

Design and implementation of data mining tool using PHP and WEKA (피에이치피와 웨카를 이용한 데이터마이닝 도구의 설계 및 구현)

  • You, Young-Jae;Park, Hee-Chang
    • Journal of the Korean Data and Information Science Society
    • /
    • v.20 no.2
    • /
    • pp.425-433
    • /
    • 2009
  • Data mining is the method to find useful information for large amounts of data in database. It is used to find hidden knowledge by massive data, unexpectedly pattern, relation to new rule. We need a data mining tool to explore a lot of information. There are many data mining tools or solutions; E-Miner, Clementine, WEKA, and R. Almost of them are were focused on diversity and general purpose, and they are not useful for laymen. In this paper we design and implement a web-based data mining tool using PHP and WEKA. This system is easy to interpret results and so general users are able to handle. We implement Apriori algorithm of association rule, K-means algorithm of cluster analysis, and J48 algorithm of decision tree.

  • PDF

Design And Implementation of a Speech Recognition Interview Model based-on Opinion Mining Algorithm (오피니언 마이닝 알고리즘 기반 음성인식 인터뷰 모델의 설계 및 구현)

  • Kim, Kyu-Ho;Kim, Hee-Min;Lee, Ki-Young;Lim, Myung-Jae;Kim, Jeong-Lae
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.12 no.1
    • /
    • pp.225-230
    • /
    • 2012
  • The opinion mining is that to use the existing data mining technology also uploaded blog to web, to use product comment, the opinion mining can extract the author's opinion therefore it not judge text's subject, only judge subject's emotion. In this paper, published opinion mining algorithms and the text using speech recognition API for non-voice data to judge the emotions suggested. The system is open and the Subject associated with Google Voice Recognition API sunwihwa algorithm, the algorithm determines the polarity through improved design, based on this interview, speech recognition, which implements the model.