• Journal of the Semiconductor & Display Technology
• /
• v.18 no.3
• /
• pp.140-143
• /
• 2019

The Internet has been expanded constantly and greatly such that we are having vast number of web pages with dynamic changes. Especially, the fast development of wireless communication technology and the wide spread of various smart devices enable information being created at speed and changed anywhere, anytime. In this situation, web crawling, also known as web scraping, which is an organized, automated computer system for systematically navigating web pages residing on the web and for automatically searching and indexing information, has been inevitably used broadly in many fields today. This paper aims to implement a prototype web crawler with Python and to improve the execution speed using threads on multicore CPU. The results of the implementation confirmed the operation with crawling reference web sites and the performance improvement by evaluating the execution speed on the different thread configurations on multicore CPU.

• The KIPS Transactions:PartD
• /
• v.13D no.5 s.108
• /
• pp.725-730
• /
• 2006

Recently Web services have become a key technology which is indispensable for e-business. Due to its ability to provide the desired information or service regardless of time and place, integrating current application systems within a single business or between multiple businesses with standardized technologies are realized using the open network and Internet. However, the current Web Services Retrieval Systems, based on text oriented search are incapable of providing reliable search results by perceiving the similarity or interrelation between the various terms. Currently there are no web services retrieval models containing such semantic web functions. This research work is purported for solving such problems by designing and implementing an extended Semantic Web Services Retrieval Model that is capable of searching for general web documents, UDDI and semantic web documents. Execution result is proposed in this paper and its efficiency and accuracy are verified through it.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.14 no.6
• /
• pp.737-744
• /
• 2004

Retrieving relevant information well corresponding to the user`s request from web is a crucial task of search engines. However, most of conventional search engines based on pattern matching schemes to queries have a limitation that is not easy to provide results corresponding to the user`s request due to the uncertainty of queries. To overcome the limitation in this paper, we propose a framework for knowledge-based semantic meta-search engines with the following five processes: (i) Query formation, (ii) Query expansion, (iii) Searching, (iv) Ranking recreation, and (v) Knowledge base. From simulation results on english-based web documents, we can see that the Proposed knowledge-based semantic meta-search engine provides more correct and better searching results than those obtained by using the Google.

• Journal of Information Science Theory and Practice
• /
• v.10 no.4
• /
• pp.3-22
• /
• 2022

This paper provides an overview of the emergence of resource discovery systems and services, along with their advantages, best practices, and current landscapes. It outlines some of the key services and functionalities of a comprehensive discovery model suitable for academic libraries in developing countries. The proposed model (VuFind as a discovery tool) performs like other existing web-scale resource discovery systems, both commercial and open-source, and is capable of providing information resources from different sources in a single-window search interface. The objective of the paper is to provide seamless access to globally distributed subscribed as well as open access resources through its discovery interface, based on a unified index. This model uses Koha, DSpace, and Greenstone as back-ends and VuFind as a discovery layer in the front-end and has also integrated many enhanced search features like Bento-box search, Geodetic search, and full-text search (using Apache Tika). The goal of this paper is to provide the academic community with a one-stop shop for better utilising and integrating heterogeneous bibliographic data sources with VuFind (https://vufind.org/vufind).

• The Journal of the Korea Contents Association
• /
• v.13 no.12
• /
• pp.575-584
• /
• 2013

Data types used for big-data analysis are very widely, such as news, blog, SNS, papers, patents, sensed data, and etc. Particularly, the utilization of web documents offering reliable data in real time is increasing gradually. And web crawlers that collect web documents automatically have grown in importance because big-data is being used in many different fields and web data are growing exponentially every year. However, existing web crawlers can't collect whole web documents in a web site because existing web crawlers collect web documents with only URLs included in web documents collected in some web sites. Also, existing web crawlers can collect web documents collected by other web crawlers already because information about web documents collected in each web crawler isn't efficiently managed between web crawlers. Therefore, this paper proposed a distributed web crawler. To resolve the problems of existing web crawler, the proposed web crawler collects web documents by RSS of each web site and Google search API. And the web crawler provides fast crawling performance by a client-server model based on RMI and NIO that minimize network traffic. Furthermore, the web crawler extracts core content from a web document by a keyword similarity comparison on tags included in a web documents. Finally, to verify the superiority of our web crawler, we compare our web crawler with existing web crawlers in various experiments.

• Journal of the Korean Society for Library and Information Science
• /
• v.48 no.2
• /
• pp.379-398
• /
• 2014

This pilot study explored the effect of mental demand on a Web searcher's thoughts, emotions, efforts, and performance in Web searches in order to address whether or not there is any difference between searchers exposed to mental demand manipulation and searchers not exposed. Research data were collected via think-aloud protocol (TAP) with a dual-task in experiments and interviews with 10 subjects who participated in this study. For the searcher's thoughts, relevance judgment was found to be hindered by mental demand. For the searcher's emotions, the experimental group was more frustrated than the control group. With respect to the searcher's efforts, searchers for the experimental group with mental demand manipulation were more likely to spend more time, make fewer queries, and visit fewer pages but work harder to find more relevant information that they needed. Lastly, with regard to the searcher's performance, it is likely that performance was highly dependent upon the completion of the search tasks for both groups. The NASA-TLX six components and cognitive load scores of searchers did not make a significant difference in the outcome. The findings support the use of a dual-task methodology as a promising approach for the assessment of cognitive load induced by complex Web searches.

• Journal of Community Nutrition
• /
• v.7 no.2
• /
• pp.107-113
• /
• 2005

This study was conducted to develop a web expert system for nutrition counseling and menu management. This program manipulates a food, dish and menu and search database that has been developed. Clients can select a recommended general and therapeutic menu using this system. The web expert system can analyze nutrients in menus and compare nutrient contents of menus with Korean Recommended Dietary Allowances. It can access the food, dish and menu database. The expert menu database can insert, store and generate the synthetic information of age, sex, and therapeutic purpose of disease. With investigation and analysis of the client's needs, the menu planning program on the internet has been continuously developed. This system consists of the database that reaches to the food composition, the dishes and the menu. Clients can search food composition and conditional food based on nutrient name and amounts. This system is able to draw up the food with its order in dish. The menu planning can be organized and nutrients analysis can be compared with Korea Recommended Allowance. This system is able to read the nutrient composition of the each food, the dish and the menu. The results of analysis is presented quickly and accurately. Therefore it can be used by not only usual people but also dietitians and nutritionists who take charge of making a menu and experts in the field of food and nutrition. It is expected that the web expert system can be useful of nutrition education, nutrition counseling and expert menu management.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.7 no.4
• /
• pp.777-782
• /
• 2012

This paper proposes a technique for improving performance using word senses and user feedback in web information retrieval, compared with the retrieval based on ambiguous user query and index. Disambiguation using query word senses can eliminating the irrelevant pages from the search result. According to semantic categories of nouns which are used as index for retrieval, we build the word sense knowledge-base and categorize the web pages. It can improve the precision of retrieval system with user feedback deciding the query sense and information seeking behavior to pages.

• Journal of KIISE:Databases
• /
• v.35 no.2
• /
• pp.192-198
• /
• 2008

It is difficult to get relevant information on vast Web data. Search engines summarize and store Web information and show the ranked lists based on user queries affected by relative importance and user-adaptation. But these have limitation with showing user-intended information at the top priority. User intention is presented in general within query itself. In this paper, we propose the selective rankup methodology of user-intended search results based on weighting internal contiguity in query terms. With experimental results, we can find user-intended results with 75.8% probability using this simple method only, and efficiency of rerank proposed outperforms ordinary case by $13{\sim}20%$.

• International Journal of Knowledge Content Development & Technology
• /
• v.8 no.3
• /
• pp.41-52
• /
• 2018

Digital Library websites plays an important role in dissemination of information of the institution and library resources. It acts as a trustworthy mirror of the institute. To evaluate the library website performance webometrics tools and indicators are required. The aim of the present research is study the webometrics of Digital Libraries of Iranian Universities of Medical Sciences on the Web to determine the amount of the visibility a website and web pages. The URL and link of 42 digital library website is obtained directly by visiting the university's website. To extract the number of indexed web pages (size), rich files have used the Google search engine Also, to extract the number of scientific resources retrieved have used the Google Scholar search engine. To calculate and obtain the number of links received have used the MOZ search engine. Generally, the results indicated that the website of Iranian digital libraries did not have a good performance in term of webometric indexes, and none of them were not rated at all indexes, only some of the websites mentioned in one or two indicators.