• Journal of Information Science Theory and Practice
• /
• v.1 no.3
• /
• pp.12-32
• /
• 2013

The paper introduces the research area of interactive information retrieval (IIR) from a historical point of view. Further, the focus here is on evaluation, because much research in IR deals with IR evaluation methodology due to the core research interest in IR performance, system interaction and satisfaction with retrieved information. In order to position IIR evaluation, the Cranfield model and the series of tests that led to the Cranfield model are outlined. Three iconic user-oriented studies and projects that all have contributed to how IIR is perceived and understood today are presented: The MEDLARS test, the Book House fiction retrieval system, and the OKAPI project. On this basis the call for alternative IIR evaluation approaches motivated by the three revolutions (the cognitive, the relevance, and the interactive revolutions) put forward by Robertson & Hancock-Beaulieu (1992) is presented. As a response to this call the 'IIR evaluation model' by Borlund (e.g., 2003a) is introduced. The objective of the IIR evaluation model is to facilitate IIR evaluation as close as possible to actual information searching and IR processes, though still in a relatively controlled evaluation environment, in which the test instrument of a simulated work task situation plays a central part.

• Journal of the Korean Society for information Management
• /
• v.14 no.2
• /
• pp.123-141
• /
• 1997

This study examined the extent to which variations in relevance assessments affect the evaluation of the performance of full-text retrieval system. Four sets of relevance judgments obtained by examining the full-text of documents were used to test the retrieval effectiveness. There was no noticeable difference in retrieval performance among the four relevance judgment sets. It implies that a variety of definitions of relevance has no effect on the evaluation of the performance of the full-text retrieval system. Furth r retrieval experiments on this topic incorporating relevance feedback, which is one of the sophisticated retrieval techniques using relevance information, are suggested.

• Journal of the Korean Society for Library and Information Science
• /
• v.16
• /
• pp.177-205
• /
• 1989

Retrieval effectiveness is the principal criteria for measuring the performance of an information retrieval system. This paper deals with the characteristics of 'relevance' of information and various measuring techniques of retrieval effectivess. The outlines of this study are as follows: 1) Relevance decision for evaluation should be devided into the user-oriented and the system-oriented decisions. 2) The recall-precision measure seems to be user-oriented, and the recall-fallout measure to be system-oriented. 3) Many of composite measures can not be justified III any rational manner unfortunately. 4) The Swets model has demonstrated that it yields, in general, a straight line instead of a curve of varying curvature and emphasized the fundamentally probabilistic nature of information retrieval. 5) The Cooper model seems to be a good substitute for precision and a useful measure for systems which ranked documents. 6) The Rocchio model were proposed for the evaluation of retreval systems which ranked documents, and were designed to be independent of cut-off. 7) The Cawkell model suggested that the Shannon's equation for entropy can be applied to measuring of retrieval effectiveness.

• Journal of the Korean Society for Library and Information Science
• /
• v.8
• /
• pp.73-101
• /
• 1981

Retrieval effectiveness is the principal criterion for measuring the performance of an information retrieval system. The effectiveness of a retrieval system depends primarily on the extent to which it can retrieve wanted documents without retrieving unwanted ones. So, ultimately, effectiveness is a function of the relevant and nonrelevant documents retrieved. Consequently, 'relevance' of information to the user's request has become one of the most fundamental concept encountered in the theory of information retrieval. Although there is at present no consensus as to how this notion should be defined, relevance has been widely used as a meaningful quantity and an adequate criterion for measures of the evaluation of retrieval effectiveness. The recall and precision among various parameters based on the 'two-by-two' table (or, contingency table) were major considerations in this paper, because it is assumed that recall and precision are sufficient for the measurement of effectiveness. Accordingly, different concepts of 'relevance' and 'pertinence' of documents to user requests and their proper usages were investigated even though the two terms have unfortunately been used rather loosely in the literature. In addition, a number of variables affecting the recall and precision values were discussed. Some conclusions derived from this study are as follows: Any notion of retrieval effectiveness is based on 'relevance' which itself is extremely difficult to define. Recall and precision are valuable concepts in the study of any information retrieval system. They are, however, not the only criteria by which a system may be judged. The recall-precision curve represents the average performance of any given system, and this may vary quite considerably in particular situations. Therefore, it is possible to some extent to vary the indexing policy, the indexing policy, the indexing language, or the search methodology to improve the performance of the system in terms of recall and precision. The 'inverse relationship' between average recall and precision could be accepted as the 'fundamental law of retrieval', and it should certainly be used as an aid to evaluation. Finally, there is a limit to the performance(in terms of effectiveness) achievable by an information retrieval system. That is : "Perfect retrieval is impossible."

• Journal of the Korean BIBLIA Society for library and Information Science
• /
• v.5 no.1
• /
• pp.85-105
• /
• 1981

Information is an essential factor leading the rapid progress which is one of the distinguished characteristics in modem society. As more information is required and as more is supplied by individuals, governmental units, businesses, and educational institutions, the greater will be the requirement for efficient methods of communication. One possibility for improving the information dissemination process is to use computers. The capabilities of such machine are beginning to be used in the process of Information storage, retrieval and dissemination. An important problems, that must be carefully examined is whether one technique for information retrieval is better for worse than another. This paper examines problem of how to evaluate an information retrieval system. One specific approach is a cost accounting model for use in studying how to minimize the cost of operating a mechanized retrieval system. Through the use of cost analysis, the model provides a method for comparative evaluation between systems. The general cost accounting model of the literature retrieval system being designed by this study are given below. 1. The total cost accounting model of the literature retrieval system. The total cost of the literature retrieval system = (the cost per unit of user time X the amount of user time) + ( the cost per unit of system time X the amount of system time) 2. System cost accounting model system cost = (the pre-search system cost per unit of time X time) + (the search system cost per unit of time X time) + (the post search system cost per unit of time X time) 1) Pre-search system cost per unit of time = cost of channel per unit time + cost of central processing unit per unit time + cost of storage per unit time 2) Search system cost per unit of time = comparison cost + document representation cost. 3) Post-search system cost per unit of time. = cost of channel per unit time + cost of central processing unit per unit time + cost of storage per unit time 3. User cost accounting model Total user cost = [pre-search user cost per unit of time X (time + additional time) ] + [search user cost per unit of time X (time + additional time) ] + [post-search user cost per unit of time X (time + additional time) ].

• Proceedings of the Korean Society for Emotion and Sensibility Conference
• /
• 2000.04a
• /
• pp.240-243
• /
• 2000

An apparel products retrieval system was proposed in which users can refer to products using Kansei evaluation values. The system adopts relevance feedback using history of the retrieval to learn the tendency of user evaluation. The system is based on a vector space retrieval model using products images expression as semantic scales. The system makes a query from user inputting information and retrieves closest products from the database. Revising algorithms of the difference method. linear multiple regression performed to investigate the effectiveness and criteria of the search. As a result of evaluation of the accuracy, it was found that the linear multiple regression and the neural network models are effective for the retrieval considering the individual Kansei.

• Journal of Advanced Marine Engineering and Technology
• /
• v.33 no.4
• /
• pp.569-577
• /
• 2009

Mobile information retrieval is an evolving branch of information retrieval that is centered on mobile and ubiquitous environments. In general, mobile devices are characterized by lightweight, low power, small memory, small display, limited input/output, low bandwidth, and so on. Some of these characteristics make it impossible to apply general information retrieval to mobile environments without any modification. In order to relieve this problem, we design and implement an information retrieval system for mobile devices like wireless phones, PDA and handheld devices. We use document summarization techniques to alleviate the limitation of small display and user profiles to retrieve the most proper documents for each individual user for personalized search. Futhermore we use meta-search to lighten some burdens visiting several portal sites. In this paper, we have implemented and demonstrated the proposed mobile information retrieval system on the domain of travel and received good evaluation from users subjectively.

• Journal of Korean Library and Information Science Society
• /
• v.40 no.3
• /
• pp.367-387
• /
• 2009

The similar bibliographic retrieval system follows whether it selects a thing of the extracted index term and or not the difference in which the similar document retrieval system There be many in the search result is generated. In this research, the method minimally making the error of the selection of the extracted candidate index term is provided In this research, the word information in which it is adjacent by using candidate index terms extracted from the similar literature and the keyword topic information were used. And by using the related author information and the reranking method of the search result, the similar bibliographic system in which an accuracy is high was developed. In this paper, we conducted experiments for similar bibliographic retrieval system on a collection of Korean journal articles of science and technology arena. The performance of similar bibliographic retrieval system was proved through an experiment and user evaluation.

• The KIPS Transactions:PartD
• /
• v.8D no.5
• /
• pp.483-494
• /
• 2001

Most of hypertext retrieval models consider documents as independent entities. They ignore relationships between documents of link semantics. in an information retrieval system for hypertext documents, retrieval effectiveness can be improved when ling information is used. Previous link-based hypertext retrieval models ignore link information while indexing. They utilize link information to re-rank the retrieval results. Therefore they are limited that only the documents is result-set utilize link information. This paper utilizes link information when indexing. We present how to use term weighting and inLinks weighting for ranking the relevant documents. Experimental results show that recall and precision evaluation according to the link semantics and the comparison with previously link_based hypertext retrieval model.

• Journal of the Korean Society for information Management
• /
• v.22 no.2 s.56
• /
• pp.41-56
• /
• 2005

For the purpose of extending the Web that is able to understand and process information by machine, Semantic Web shared knowledge in the ontology form. For exquisite query processing, this paper proposes a method to use semantic relations in the ontology as relevance feedback information to query expansion. We made experiment on pharmacy domain. And in order to verify the effectiveness of the semantic relation in the ontology, we compared a keyword based document retrieval system that gives weights by using the frequency information compared with an ontology based document retrieval system that uses relevant information existed in the ontology to a relevant feedback. From the evaluation of the retrieval performance. we knew that search engine used the concepts and relations in ontology for improving precision effectively. Also it used them for the basis of the inference for improvement the retrieval performance.