Browse > Article
http://dx.doi.org/10.14400/JDC.2017.15.11.261

Crepe Search System Design using Web Crawling  

Kim, Hyo-Jong (Department of Information Security, Tongmyong University)
Han, Kun-Hee (Division of Information & Communication Engineering, Baekseok University)
Shin, Seung-Soo (Department of Information Security, Tongmyong University)
Publication Information
Journal of Digital Convergence / v.15, no.11, 2017 , pp. 261-269 More about this Journal
Abstract
The purpose of this paper is to provide a search system using a method of accessing the web in real time without using a database server in order to guarantee the up-to-date information in a single network, rather than using a plurality of bots connected by a wide area network Design. The method of the research is to design and analyze the system which can search the person and keyword quickly and accurately in crepe system. In the crepe server, when the user registers information, the body tag matching conversion process stores all the information as it is, since various styles are applied to each user, such as a font, a font size, and a color. The crepe server does not cause a problem of body tag matching. However, when executing the crepe retrieval system, the style and characteristics of users can not be formalized. This problem can be solved by using the html_img_parser function and the Go language html parser package. By applying queues and multiple threads to a general-purpose web crawler, rather than a web crawler design that targets a specific site, it is possible to utilize a multiplier that quickly and efficiently searches and collects various web sites in various applications.
Keywords
Digital Curation; Contents; Web Crawler; Search system; Keyword search; Module;
Citations & Related Records
Times Cited By KSCI : 9  (Citation Analysis)
연도 인용수 순위
1 Jung-In Kim, Byung-Man Kim, Jung-Ju Kim, "A Development of Digital Curation System for Creativity and Personality Education", Journal of Korea Multimedia Society, Vol. 19, No. 9, pp.1710-1722, 2016.   DOI
2 Young-Hee Ahn, Ok-Wha Park, "Development of a Framework for Digital Curation Policy", Journal of Korean Library and Information Science Society, Vol 41, No. 1, pp.167-186, 2010.   DOI
3 Kang Soon Lee, "Development of Elementary Dance Education Program Using ICT", Korean Society For The Study Of Physical Education, Vol. 18, No. 2, pp.77-89, 2013.
4 H.K. Kim, Digital Curation Framework Research for Analyzing Issues Based on Big- Data, Master's Thesis of Chung-Ang University of Technology, 2014.
5 Jung-In Kim, Byung-Man Kim, Jung-Ju Kim, "A Development of Digital Curation System for Creativity and Personality Education", Journal of Korea Multimedia Society, Vol. 19, No. 9, pp. 1710-1722, 2016.   DOI
6 S.S. Shin, J.I. Kim, and J.J. Youn, "Vulnerability Analysis of the Creativity and Personality Education Based on Digital Convergence Curation System," Journal of Korea Convergence Society, Vol. 6, No. 4, pp.225-234, 2015.   DOI
7 Kwang-Young Kim, Won-Goo Lee, Hwa-Mook Yoon, Sung-Ho Shin, Min-Ho Lee, "Development of Web Crawler for Archiving Web Resources," Journal of the Korea Contents Association, Vol. 11, No. 9, pp.9-16, 2011.   DOI
8 Wan-Sup Cho, Jeong-Eun Lee, Chi-Hwan Choi, "Refresh Cycle Optimization for Web Crawlers," Journal of the Korea Contents Association, Vol. 13, No. 6, pp.30-39, 2013.   DOI
9 H.H. Lee and W.J. Lee, "A Study on the Design of Curation System of Customized Sport Convergence Contents for Activation of Sport for All," Journal of Korea Multimedia Society, Vol. 19, No. 2, pp. 396-404, 2016.   DOI
10 N.E. Han and S.H. Kim, "Comparative Analysis on Digital Curation Process in Foreign Academic Libraries," Journal of Korean Library and Information Science Society, Vol. 45, No. 2, pp. 93-116, 2014.
11 B.H. Cho, "The Trend of Digital Curation Service," Week Technology Trends, Vol. 2013, No. 42, pp. 1-10, 2013.
12 Myoung-sil Choi , "A Study on the Improvement of the Web-Crawler Performance based on Weighted Directed Graph," Department of Computer Science, Graduate School, Kyungpook National University, 2010.
13 Dae Yu Kim, Jung Tae Kim, "Efficient Design of Web Searching Robot Engine Using Distributed Processing Method with Javascript Function," The journal of the Korea Institute of Maritime Information & Communication Sciences, Vol. 13, No. 12, pp.2595-2602, 2009.
14 Kwang Hyun Kim, Joon Ho Lee, "A Methodology for Performance Evaluation of Web Robots," Information Processing Society, Vol. 11, No. 3, pp.563-570, 2006.