• Science of Emotion and Sensibility
• /
• v.12 no.4
• /
• pp.489-500
• /
• 2009

This research takes a user-centered approach to define relevance, the core concept in information retrieval. The literature on relevance has identified numerous factors affecting such a judgment. We examined the model of user relevance judgment that describes the relationship between user relevance criteria and different types of relevance with information search task. We consider 7 criteria of user relevance-topicality, novelty, reliability, understandability, specificity, richness, and interest-and 3 type of user relevance-cognitive relevance, situational relevance, and affective relevance. Data were collected from a semi-controlled survey and analyzed by a structural equation modeling. As a result, topicality and reliability were found to be the essential relevance criteria in all information retrieval tasks. In the fact search task, topicality, reliability, novelty, richness, and interest were found to be significant. In the problem solving search task, topicality, reliability, understandability, and specificity were found to be significant. In the decision making search task, topicality, reliability, novelty, understandability, richness, specificity, and interest were found to be significant. In addition, the relationships between types of user relevance were determined. This research made theoretical and practical contributions to the field of information retrieval by identifying a definite model of user relevance judgment.

• Journal of the Korean Society for information Management
• /
• v.32 no.3
• /
• pp.277-293
• /
• 2015

This study examined the relative retrieval effectiveness after relevance feedback between two systems (Title/Abstract and Full-text) using four different sets of relevance judgment. Four relevance levels (not relevant, marginally relevant, relevant, highly relevant) are also used, each of which is determined by referees giving a relevance score to documents. This study also investigated how much the average precision was improved after relevance feedback when "marginally relevant" documents are included in the relevant class with the Title/Abstract system, and with the Full-text retrieval system as well. It is found that the Title/Abstract system benefited from relevance feedback with the marginally relevant documents. In case of the Title/Abstract system, the higher percentage of improvement was consistently obtained when including the marginally relevant documents in the relevance class, however the result was vice versa in case of the Full-text retrieval system. It implied that the marginally relevant documents in the relevant class had caused noises in the Full-text retrieval system.

• Journal of the Korean Society for information Management
• /
• v.14 no.2
• /
• pp.123-141
• /
• 1997

This study examined the extent to which variations in relevance assessments affect the evaluation of the performance of full-text retrieval system. Four sets of relevance judgments obtained by examining the full-text of documents were used to test the retrieval effectiveness. There was no noticeable difference in retrieval performance among the four relevance judgment sets. It implies that a variety of definitions of relevance has no effect on the evaluation of the performance of the full-text retrieval system. Furth r retrieval experiments on this topic incorporating relevance feedback, which is one of the sophisticated retrieval techniques using relevance information, are suggested.

• ETRI Journal
• /
• v.43 no.5
• /
• pp.856-868
• /
• 2021

Urdu is a widely spoken language in the Indian subcontinent with over 300 million speakers worldwide. However, linguistic advancements in Urdu are rare compared to those in other European and Asian languages. Therefore, by following Text Retrieval Conference standards, we attempted to construct an extensive text collection of 85 304 documents from diverse categories covering over 52 topics with relevance judgment sets at 100 pool depth. We also present several applications to demonstrate the effectiveness of our collection. Although this collection is primarily intended for text retrieval, it can also be used for named entity recognition, text summarization, and other linguistic applications with suitable modifications. Ours is the most extensive existing collection for the Urdu language, and it will be freely available for future research and academic education.

• Journal of the Korean Society for Library and Information Science
• /
• v.50 no.1
• /
• pp.417-435
• /
• 2016

Relevance judgment is important in terms of improving the effectiveness of information retrieval systems, and it has been dominant for users to search and use images utilizing internet and digital technologies. However, in the field of image retrieval, there have been only a few studies in terms of identifying relevance criteria. The purpose of this study aims to identify and characterize the non-relevance criteria from the failed image searches. In order to achieve the purpose of this study, a total of 135 participants were recruited and a total of 1,452 criteria items were collected for this study. Analyses and identification on the data set found thirteen criteria such as 'topicality', 'visual content', 'accuracy', 'visual feature', 'completeness', 'appeal to user', 'focal point', 'bibliographic information', 'impression', 'posture', 'face feature', 'novelty', and 'time frame'. Among these criteria, 'visual content' and 'focal point' were introduced in this current study, while 'action' criterion identified in previous studies was not shown in this current study. When image needs and image uses are analyzed with these criteria, there are distinctive differences depending on different image needs and uses.

• Journal of the Korean Data Analysis Society
• /
• v.20 no.6
• /
• pp.3003-3014
• /
• 2018

This paper investigates whether the value relevance of unbilled revenue in the construction company is differentiated according to the degree of company's operational efficiency. Unbilled revenue is an item that appears in the assets of the statement of financial position based on the IFRS. It is an item that has a nature of accounts receivable, but it is calculated based on subjective judgment of management. In this paper, we hypothesized that the value relevance of unbilled revenue will be differentiated according to operational efficiency of the construction company. The results of hypothesis test are as follows. First, unbilled revenue has a significant relationship with firm's value, which is consistent with previous research results. Second, the value relevance of unbilled revenue is higher in companies with high operational efficiency. In conclusion, this paper is different from previous researches in that this paper presents not only that unbilled revenue has a value relevance but also provides evidence that it can be differentiated according to the degree of operational efficiency.

• Journal of the Korean Society for information Management
• /
• v.23 no.2
• /
• pp.97-111
• /
• 2006

The purpose of this study is to investigate what level of difference in precision would be significantly perceived by a human user of an information retrieval system. Not many researches have been conducted with regards to this issue in information retrieval field. Despite the non-significant results, there were several interesting findings in recognizing different levels of precision rates. The correctness of relevance task had little to do with the taken time for the task. In addition, the strong relationship between the subjects' topic familiarity and rate of correct judgments is one of the most interesting results in this study. It turned out that the subjects have more difficulty in a situation they have to judge between the two lists having more non-relevant documents than in a situation they do between the lists haying more relevant documents. Finally, the serious influence from the first top N documents in a list for relevance judgment task has been confirmed.

• Journal of the Korean Society for information Management
• /
• v.11 no.1
• /
• pp.81-102
• /
• 1994

Accordmg to the development of various information retneval system suitable for Korean database, many researchers have realized the need of R Test ColleAon which can be r d y used for evaluatmg a retneval system. Therefore, This study developed the TEST SET whch helps ob&vely evaluatmg the retrieval performance of an Hangul Automatic Indexer or Korean Information Retrieval System. The developed Test Set has four files such as: 1) Korean Document Set( * . all): 2) Natural Language Query Set(KTsetnq1): 3) Boolean Query Set(Ktset.bq1): 4) Query-Relevance Judgment Set ( KTsetrel) .

• Journal of the Korean Society for Library and Information Science
• /
• v.48 no.2
• /
• pp.379-398
• /
• 2014

This pilot study explored the effect of mental demand on a Web searcher's thoughts, emotions, efforts, and performance in Web searches in order to address whether or not there is any difference between searchers exposed to mental demand manipulation and searchers not exposed. Research data were collected via think-aloud protocol (TAP) with a dual-task in experiments and interviews with 10 subjects who participated in this study. For the searcher's thoughts, relevance judgment was found to be hindered by mental demand. For the searcher's emotions, the experimental group was more frustrated than the control group. With respect to the searcher's efforts, searchers for the experimental group with mental demand manipulation were more likely to spend more time, make fewer queries, and visit fewer pages but work harder to find more relevant information that they needed. Lastly, with regard to the searcher's performance, it is likely that performance was highly dependent upon the completion of the search tasks for both groups. The NASA-TLX six components and cognitive load scores of searchers did not make a significant difference in the outcome. The findings support the use of a dual-task methodology as a promising approach for the assessment of cognitive load induced by complex Web searches.

• International Journal of Highway Engineering
• /
• v.14 no.1
• /
• pp.1-8
• /
• 2012

The horizontal alignment of road is deeply related with the stability of the road and traffic capacity. It is necessary to analyze horizontal alignment of road accurately for efficient maintenance of the road and relevance judgment about the standard. Recently the study on horizontal alignment of road using Lidar data and GPS was concluded, but they were many problem analyzing horizontal alignment radius of curvature in wide area. In this study, the tool which the radius of curvature can evaluate the suitability about "Rules about the Road Structure & Facilities Standards" by using center lines of the road of the digital map tries to implement on GIS. The interface was designed and implemented which can automatically estimate the Road Centerline Horizontal Alignment by using $ESRI^{(R)}$ $ArcObject^{TM}$.