• Journal of Information Science Theory and Practice
• /
• v.1 no.3
• /
• pp.12-32
• /
• 2013

The paper introduces the research area of interactive information retrieval (IIR) from a historical point of view. Further, the focus here is on evaluation, because much research in IR deals with IR evaluation methodology due to the core research interest in IR performance, system interaction and satisfaction with retrieved information. In order to position IIR evaluation, the Cranfield model and the series of tests that led to the Cranfield model are outlined. Three iconic user-oriented studies and projects that all have contributed to how IIR is perceived and understood today are presented: The MEDLARS test, the Book House fiction retrieval system, and the OKAPI project. On this basis the call for alternative IIR evaluation approaches motivated by the three revolutions (the cognitive, the relevance, and the interactive revolutions) put forward by Robertson & Hancock-Beaulieu (1992) is presented. As a response to this call the 'IIR evaluation model' by Borlund (e.g., 2003a) is introduced. The objective of the IIR evaluation model is to facilitate IIR evaluation as close as possible to actual information searching and IR processes, though still in a relatively controlled evaluation environment, in which the test instrument of a simulated work task situation plays a central part.

• Journal of the Korean Society for information Management
• /
• v.14 no.2
• /
• pp.123-141
• /
• 1997

This study examined the extent to which variations in relevance assessments affect the evaluation of the performance of full-text retrieval system. Four sets of relevance judgments obtained by examining the full-text of documents were used to test the retrieval effectiveness. There was no noticeable difference in retrieval performance among the four relevance judgment sets. It implies that a variety of definitions of relevance has no effect on the evaluation of the performance of the full-text retrieval system. Furth r retrieval experiments on this topic incorporating relevance feedback, which is one of the sophisticated retrieval techniques using relevance information, are suggested.

• Proceedings of the Korean Society for Emotion and Sensibility Conference
• /
• 2000.04a
• /
• pp.240-243
• /
• 2000

An apparel products retrieval system was proposed in which users can refer to products using Kansei evaluation values. The system adopts relevance feedback using history of the retrieval to learn the tendency of user evaluation. The system is based on a vector space retrieval model using products images expression as semantic scales. The system makes a query from user inputting information and retrieves closest products from the database. Revising algorithms of the difference method. linear multiple regression performed to investigate the effectiveness and criteria of the search. As a result of evaluation of the accuracy, it was found that the linear multiple regression and the neural network models are effective for the retrieval considering the individual Kansei.

• Journal of Information Science Theory and Practice
• /
• v.3 no.4
• /
• pp.19-34
• /
• 2015

Evaluation of retrieval systems is mostly limited to laboratory settings and rarely considers changes of performance over time. This article presents an evaluation of retrieval systems for internal Web site search systems between the years 2006 and 2011. A holistic evaluation methodology for real Web sites was developed which includes tests for functionality, search quality, and user interaction. Among other sites, one set of 20 Web site search systems was evaluated three times in different years and no substantial improvement could be shown. It is surprising that the communication between site and user still leads to very poor results in many cases. Overall, the quality of these search systems could be improved, and several areas for improvement are apparent from our evaluation. For a comparison, Google’s site search function was also tested with the same tasks.

• Journal of the Korean Society for Library and Information Science
• /
• v.16
• /
• pp.177-205
• /
• 1989

Retrieval effectiveness is the principal criteria for measuring the performance of an information retrieval system. This paper deals with the characteristics of 'relevance' of information and various measuring techniques of retrieval effectivess. The outlines of this study are as follows: 1) Relevance decision for evaluation should be devided into the user-oriented and the system-oriented decisions. 2) The recall-precision measure seems to be user-oriented, and the recall-fallout measure to be system-oriented. 3) Many of composite measures can not be justified III any rational manner unfortunately. 4) The Swets model has demonstrated that it yields, in general, a straight line instead of a curve of varying curvature and emphasized the fundamentally probabilistic nature of information retrieval. 5) The Cooper model seems to be a good substitute for precision and a useful measure for systems which ranked documents. 6) The Rocchio model were proposed for the evaluation of retreval systems which ranked documents, and were designed to be independent of cut-off. 7) The Cawkell model suggested that the Shannon's equation for entropy can be applied to measuring of retrieval effectiveness.

• Journal of Korea Society of Digital Industry and Information Management
• /
• v.17 no.3
• /
• pp.17-26
• /
• 2021

Internet of Things is an infrastructure of the interconnected devices. In Internet of Things environment, many smart devices are used in daily life. It requires a new retrieval method using multiple devices. We propose an information retrieval method using both device characteristics and device usage characteristics in multi-device environments. Firstly, information retrieval is performed using a general purpose device. And then, it is performed using dedicated devices. Our method uses both characteristics of the devices and usage characteristics of them. Moreover, it considers queries on the general purpose device. This paper proposes a new retrieval method and describes algorithms. Then, it presents smart home scenarios. Performance evaluation is performed using the scenarios. The evaluation results show higher precision and efficiency than previous researches. The proposed method gets information more accurately and quickly in IOT multiple device environments.

• The Journal of the Acoustical Society of Korea
• /
• v.26 no.3E
• /
• pp.74-79
• /
• 2007

In this paper, we report our recent work on a indexing and retrieval system of broadcast news using audio content analysis. Key issues addressed in this work are two major parts of the audio indexing system: anchorperson detection based on audio segmentation, and phone-based spoken document retrieval, developed in the framework of the emerging MPEG-7 standard. Experiments are conducted on a database of Britisch broadcast news videos. We discuss the development of the retrieval system, and the evaluation of each part and the retrieval system.

• Journal of the Korean Society for information Management
• /
• v.21 no.2
• /
• pp.189-210
• /
• 2004

The purpose of this study is to evaluate the performance of XML retrieval based on ORDBMSs(Object-Relational Database Management Systems) approach. This paper describes indexing and retrieval methods for XML documents and the methodologies of experiments at INEX(Initiative for the Evaluation of XML retrieval). Like any other traditional information retrieval experiment, the test collection was consists of documents, topics/queries, task, relevance assessments and evaluation. EXIMA$^{TM}$ Supply, a kind of native XML DB based on ORDBMS technologies, is used for this experiment. Although this approach has many benefits, for example, no delay in storing and searching XML documents. but it showed relatively disappointed retrieval performance at INEX 2002. This result may caused since the given topics had to be decomposed and modified to be processed by the XPath processor, and during this modification the original meaning of topics can be changed inevitably and some important information nay pass over.r.

• Journal of the Korean BIBLIA Society for library and Information Science
• /
• v.22 no.2
• /
• pp.141-160
• /
• 2011

This study suggested knowledge base and search engine for the libraries that have the largescaled data. For this purpose, 3 components of knowledge bases(triple ontology, concept-based knowledge base, inverted file) were constructed and 3 search engines(search engine JENA for rule-based reasoning, Concept-based search engine, keyword-based Lucene retrieval engine) were implemented to measure their performance. As a result, concept-based retrieval engine showed the best performance, followed by ontology-based Jena retrieval engine, and then by a normal keyword search engine.

• Journal of Korean Society of Archives and Records Management
• /
• v.13 no.3
• /
• pp.67-97
• /
• 2013

The purpose of this study is to design and evaluate an individual instance-based ontology retrieval system for archival records of the "Saemaul Movement". We used Protege editor 4.1 to design an individual instance-based ontology. To evaluate the proposed ontology retrieval system, five short queries and ten narrative queries were used and compared their precision and recall against the NARA keyword-based retrieval system. The performance results showed that the individual-based ontology retrieval system outperformed the keyword-based retrieval system in terms of the measurement of precision and recall.