Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Design and Adaptation for Internet News Data Extraction Middleware(INDEM) System

  • Sun, Bok-Keun (Dept. of Computer Engineering, Hoseo University)
  • Received : 2016.02.25
  • Accepted : 2016.03.22
  • Published : 2016.04.29
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose the INDEM(Internet News Data Extraction Middleware) system for the removal of the unnecessary data in internet news. Although data on the internet can be used in various fields such as source of data of IR(Information Retrieval), Data mining and knowledge information service, it contains a lot of unnecessary information. The removal of the unnecessary data is a problem to be solved prior to the study of the knowledge-based information service that is based on the data of the web page. The INDEM system parses html and explores the XPath, and it is to perform the analysis. The user simply utilize INDEM by implementing an abstract class that provides INDEM, and can obtain the analysis information. INDEM System through this process delivers the analysis information including the main contents of news site to the users. In this paper, the INDEM system was adapted in a stand-alone and web service system and it was evaluated on the basis of 16 news site. As a result, performance of the INDEM system is affected in html source data size and complexity of used html grammar than the main news data size.

Keywords

References

  1. B. K. Sun, "A Study of Main Contents Extraction from Web News Pages based on XPath Analysis", Journal of The Korea Society of Computer and Information, Vol. 20, No. 7, pp. 1-7, July 2015. https://doi.org/10.9708/jksci.2015.20.7.001
  2. J. Si, W. Wang, "A Template-based forum posts content extraction method", International Conference on ICECE, pp.38-41, 2011.
  3. R. Gunasundari, S. Karthikeyan, "A Study of content extraction from web pages based on links", International Journal of Data Mining & Knowledge management Process(IJDKP) vol.2, No.3, May 2012.
  4. B. Zhou, C. Wang, Q. Su, "Chinese web page content extraction based on page content analysis", Journal of Computational Information Systems vol.5, No.6, pp.1861-1871, Dec 2009.
  5. S.Pretzsch, K.Muthmann, A.Schill, "FODEX-Towards generic data extraction from web forums", 26th International conference on advanced information networking and applications workshops, pp.821-826, 2012.
  6. Clearly, https://chrome.google.com/webstore/detail/clearly/iooicodkiihhpojmmeghjclgihfjdjhj
  7. Readability, https://www.readability.com/
  8. S.Gupta, G. Kaiser, D. Neistadt, and P. GS.Gupta, G. Kaiser, D. Neistadt, and P. Grimm, "DOM-based content extraction of HTML documents", in WWW '03: Proceedings of the 12th International Conference on WWW, ACM, pp.207-214, 2003.
  9. N. Negm, P. Elkafrawy, A.B. Salem, "A Survey of Web Information Extraction Tools", International Journal of Computer Applications, Vol. 43, No. 7, pp.19-27, April 2012. https://doi.org/10.5120/6115-8296
  10. H. Mohammadzadeh, T. Gottron, F. Schweiggert, G. Nakhaeiza, "A Fast and accurate approach for main content extraction based on character encoding", 22nd International workshop on database and expert systems applications, pp.167-171. 2011.
  11. SY. Oh, "X2RD: Storing and Quering XML Data Using XPath to Relational Database", Journal of The Korea Society of Computer and Information, Vol. 14, No. 3, pp. 57-64, March 2009.
  12. XPath, http://www.w3.org/TR/xpath/