• Journal of the Korea Computer Industry Society
• /
• v.3 no.3
• /
• pp.331-340
• /
• 2002

The web world is getting so huge and untractable that without an intelligent information extractor we would get more and more helpless. Conventional web spidering techniques for general purpose search engine may be too slow for the specific search engines, which concentrate only on specific areas or keywords. In this paper a new model for improving web spidering capabilities is suggested and experimented. How to select adequate reference web pages from the initial web Page set relevant to a given specific area (or keywords) can be very important to reduce the spidering speed. Our reference web page selection method DOPS dynamically and orthogonally selects web pages, and it can also decide the appropriate number of reference pages, using a newly defined measure. Even for a very specific area, this method worked comparably well almost at the level of experts. If we consider that experts cannot work on a huge initial page set, and they still have difficulty in deciding the optimal number of the reference web pages, this method seems to be very promising. We also applied reinforcement learning to web environment, and DOPS-based reinforcement learning experiments shows that our method works quite favorably in terms of both the number of hyper links and time.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.15 no.4
• /
• pp.395-399
• /
• 2005

In this paper, we presents HIGH-Q learning algorithm with reinforcement learning for more fast and exact topic-directed web spidering. The purpose of reinforcement learning is to maximize rewards from environment, an reinforcement learning agents learn by interacting with external environment through trial and error. We performed experiments that compared the proposed method using reinforcement learning with breath first search method for searching the web pages. In result, reinforcement learning method using future discounted rewards searched a small number of pages to find result pages.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2001.12a
• /
• pp.259-262
• /
• 2001

본 논문에서는 사용자의 취향에 맞춰 특정 웹 문서를 탐색하는 개인화된 웹 검색기의 구현을 다룬다. 사용자의 취향은 사용자의 직접적인 평가와 사용자의 검색 과정을 통해 얻어지는 간접적인 평가를 사용한 강화 학습을 사용하여 학습된다. 웹 문서의 검색은 사용자의 취향과 현재 문서와의 관련 도를 보상으로 사용한 강화 학습을 통하여 이루어진다.