Browse > Article
http://dx.doi.org/10.3745/KTSDE.2017.6.5.279

Logistic Regression Ensemble Method for Extracting Significant Information from Social Texts  

Kim, So Hyeon (서울시립대학교 전자전기컴퓨터공학과)
Kim, Han Joon (서울시립대학교 전자전기컴퓨터공학과)
Publication Information
KIPS Transactions on Software and Data Engineering / v.6, no.5, 2017 , pp. 279-284 More about this Journal
Abstract
Currenty, in the era of big data, text mining and opinion mining have been used in many domains, and one of their most important research issues is to extract significant information from social media. Thus in this paper, we propose a logistic regression ensemble method of finding the main body text from blog HTML. First, we extract structural features and text features from blog HTML tags. Then we construct a classification model with logistic regression and ensemble that can decide whether any given tags involve main body text or not. One of our important findings is that the main body text can be found through 'depth' features extracted from HTML tags. In our experiment using diverse topics of blog data collected from the web, our tag classification model achieved 99% in terms of accuracy, and it recalled 80.5% of documents that have tags involving the main body text.
Keywords
Machine Learning; Information Extraction; Ensemble; Logistic Regression; Social Media;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Jung-hwan Bae, Ji-eun Son, and Min Song, "Analysis of Twitter for 2012 South Korea Presidential Election by Text Mining Techniques," Journal of Intelligence and Information Systems, Vol.19, No.3, pp.141-156, 2013.   DOI
2 Yoon-Ju Lee, Ji-Joon Seo, and Jin-Tak Choi, "Fashion Trend Marketing Prediction Analysis Based on Opinion Mining Applying SNS Text Contents," Journal of Korean Institute of Information Technology (KIIT), Vol.12, No.12, pp.163-170, 2014.
3 Imran, Muhammad et al., "Extracting information nuggets from disaster-related messages in social media," Proc. of ISCRAM, Baden-Baden, Germany, 2013.
4 So-hyeon Kim and Han-joon Kim, "Extracting Significant Information from Social Text using Machine Learning," Korea Information Processing Society, The KIPS Fall Conference, Vol.23, No.2, pp.742-745, 2016.
5 Wang, Changzhi et al., "Opinion Mining Research on Chinese Micro-blog," First International Conference on Information Science and Electronic Technology, 2015.
6 Gulhane, Pankaj et al., "Exploiting content redundancy for web information extraction," Proceedings of the VLDB Endowment, Vol.3, pp.578-587, 2010.
7 Bronzi, Mirko et al., "Extraction and integration of partially overlapping web sources," Proceedings of the VLDB Endowment, Vol.6, No.10, pp.805-816, 2013.
8 Kohlschütter, Christian, Peter Fankhauser, and Wolfgang Nejdl, "Boilerplate detection using shallow text features," Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp.441-450, 2010.
9 Tomaz K, Evaluating Text Extraction Algorithms [Internet], http://tomazkovacic.com/blog/.
10 Sun, Fei, Dandan Song, and Lejian Liao, "Dom based content extraction via text density," Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.245-254, 2011.
11 Narawade, Shubhada Maruti et al., "A Web Based Data Extraction Using Hierarchical (DOM) Tree Approach," International Journal for Innovative Research in Science and Technology, Vol.2, No.11, pp.255-257, 2016.
12 Geng, Hua, Qiang Gao, and Jingui Pan, "Extracting content for news web pages based on DOM," IJCSNS International Journal of Computer Science and Network Security, Vol.7, No.2, pp.124-129, 2007.
13 Kadam, Vinayak B., and Ganesh K. Pakle, "DEUDS: Data Extraction Using DOM Tree and Selectors," International Journal of Computer Science and Information Technologies, Vol.5, No.2, pp.1403-1410, 2014.
14 Kuswanto, Heri et al., "Logistic Regression Ensemble for Predicting Customer Defection with Very Large Sample Size," Procedia Computer Science, Vol.72, pp.86-93, 2015.   DOI
15 Wang, Hong, Qingsong Xu, and Lifeng Zhou, "Large unbalanced credit scoring using Lasso-logistic regression ensemble," PloS one, Vol.10, No.2, e0117844, 2015.   DOI
16 Chandrashekar, Girish, and Ferat Sahin, "A survey on feature selection methods," Computers & Electrical Engineering, Vol.40, No.1, pp.16-28, 2014.   DOI
17 Jurado, Sergio et al., "Hybrid methodologies for electricity load forecasting: Entropy-based feature selection with machine learning and soft computing techniques," Energy, Vol.86, pp.276-291, 2015.   DOI