Journal of the Korean Society for information Management (정보관리학회지)

Korean Society for Information Management (한국정보관리학회)
권호 표지

A Theoretical Review of Relevance Judgments

적합 판단 영향 요인에 관한 이론적 고찰

  • Published : 1996.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

Relevance judgments play a very important role in evaluation of information systems since the degree of success of the information retrieval depends on the relevance judgments. This article reviews the theoretical background of the concept of 'relevance' associated with information retrieval evaluation and tries to identify whether there is any factor that affects relevance judgments. By reviewing previous researches done in the information retrieval evaluation field, four variables have been identified as impacting factors, such as document surrogates presented to judges, the order of presentation, measuring devices of relevance judgments and judges.

정보검색시스템평가에서 적합 판단(relevance judgments)은 중요한 역할을 수행한다. 그 이유는 적합 판단의 결과에 따라 탐색성능이 결정되기 때문이다. 본 논문은 적합 판단과 관련하여 적합 개념의 이론적 배경을 역사적으로 고찰하고 적합개념이 적합판단에 미친 영향을 살펴보고자 한다. 또한 적합 판단과정에서 판단에 영향을 주는 요인이 있는지를 조사하고자 한다. 선행연구는 문들에 관한 문헌조사연구 결과는 적합 판단에 영향을 미치는 변인으로 다음의 네 종류의 변인이 파악되었다. 즉, 적합 판단근거로 제시되는 문헌대용물의 종류, 제시순서, 적합측정도구, 판단자 변인의 네 변인들은 적합 판단에 영향을 미치는 것으로 확인되었다.

Keywords

References

  1. Proceedings of the American Documentation Institute A comparison of relevance assessments by three types of evaluator Barhydt,G.C.
  2. Jr. of Documentation v.23 no.1 The effectiveness of non-user relevance assessments Barhydt,G.C.
  3. Proceeding of the ASIS Annual Meeting v.30 A preliminary examination of clues to relevance criteria within document representations Barry, Carol L.
  4. Jr. of American Society for Information Science v.45 no.3 User-defined relevance criteria an exploratory study Barry, Carol L.
  5. Ph.D. dissertation Some Attributes of Online Search Intermediaries That Relate to Search Outcome Bellardo,Trudi.
  6. Information Processing and Management v.18 Beyond topicality;A two stage view of relevance and the retrieval process Boyce
  7. Information Processing and Management v.19 Relatedness relevance and responsiveness in retreival systems Buckland,M.K.
  8. Information Processing and Management v.27 The effect of indexing exhaustivity on retrieval performance Burgin,R.
  9. Information Processing and Managements v.28 no.5 Variations in relevance judgments and the evaluation of retrievel performance Burgin,Robert.
  10. ASLIB Cranfield Research Project:Factors Determining the Performance of Indexing Systems v.2 Cleverdon,Cyril.;Mills,J.;Keen,M.
  11. Information Storage and Retrieval v.7 A definition of relevance for information retrieval Cooper,W.S.
  12. Jr. of American Society for Information Science v.24 On Selecting a measure of retrieval effectiveness Cooper,W.S.
  13. Description of Individual Studies v.2 Experimental Studies of Relevance Judgments:Final Report Cuadra,Carlos A.;Katter, Robert V.
  14. Jr. of Documentation v.23 no.4 Opening the black box of relevance Cuadra,Carlos A.;Katter, Robert V.
  15. Automation and Sciencetific Communication, Short paper Is Relevance and Adequate Criterion for Retrieval System Evaluation? Doyle,L.B.;H.P.Luhn(ed.)
  16. Ph.D. Dissertation. Syracuse University Magnitude Estimation and the Measurement of Relevance Eisenberg,M.B.
  17. Information Processing and Management v.24 no.4 Measuring relevance judgments Eisenberg,Michael.
  18. Jr. of American Society for Information Science v.39 Order effects:A Study of the possible influence of presentation order on user judgments of document relevance Eisenberg,Michael.;Barry,Carol
  19. ASIS Mid-Year Proceedings Relavance:The Search for a definition Eisenberg,Michael.;Schamber,Linda.
  20. Jr. of Information Science v.8 Theory and explanation in information retrieval research Ellis,David.
  21. Ph.D. Dissertation. Drexel University Online Information Retrieval:Identification of Measures That Discriminate Among Users with Different Levels of Types of Experience Fenchel,Carol H.
  22. Information Storage and Retrievel v.8 no.2 A note on the concept of relevance Foskett,D.J.
  23. Psychophsics:Method. theory and application(2nd ed) Gescheider,G.
  24. Information Processing and Management v.32 no.1 The exploring the relationship between user satisfaction and relevance in information systems Gluck,M.
  25. American Documentaion v.7 Seven years of work on the organization of materials in the special library Gull,C.D.
  26. Library Quarterly v.41 no.3 The Cranfield Ⅱ relevance assessments:A critical evaluation Harter,Stephen P.
  27. Jr. of American Society for Information Science v.43 no.9 Psychological relevance and informatio science Harter,Stephen P.
  28. Ph. D dissertation. Syracuse University Towards a Search Theory of Information Janes,J.w.
  29. Information Processing and Management v.27 no.6 Relevance judgments and the incremental presentation of document representation Janes,Joseph W.
  30. Library Quarterly v.62 no.2 Relevance judgements of actual users and secondary judges:A comparative Study Janes,Joseph W.;Mckinney,Renee
  31. Proceedings of the ASIS Toward a More Realistic Assessment of Information Retrieval Performance Katzer,J.;Snyder,H.
  32. Information Storage and Retrieval v.10 no.1 Relevance, pertinence and information system development Kemp,D.A.
  33. American Documentation v.20 MEDLARS:Report on its operation efficiency Lancaster,F.W.
  34. Information Storge and Retrieval v.4 no.3 Relevance assessment and retrieval system evaluation Lesk,M.E.;G.Salton
  35. Jr. of American Society for Information Science v.29 Catalog information and text as indicators of relevance Marcus,R.S.;Kugel,P.;Benenfeld,A.R.
  36. Jr. of the American Society for Information Science v.36 Relevance? Meadow, Charles T.
  37. American Documentation v.18 no.3 Relevance disagreement and unclear request forms O'Connor,J.
  38. American Documentation v.20 no.4 Some independent agreements and resolved disagreements about snswer-providing documetns O'Connor,J.
  39. Library Quarterly v.68 no.3 The nature of relevance in information retrieval:An empirical Study Park, Taemin Kim
  40. Jr. of American Society for Information Science v.41 no.7 Does order of presentation affect users' judgment of document? Parker, Lorraine M. Purgailis;Johnson, Robert E.
  41. American Documentation v.12 no.2 Comparisons of four types of lexical indicators of content Rath,G.J.;A.Resnick;Savage,T.R.
  42. Annual Review of Information Science and Technology v.2 Evaluation of information systems and services Rees,A.M.
  43. Final Report. Center for Documentation and Communication Research A Field Experimental Approach to the Study of Relevance Assessments in Relation to Document Searching Rees,A.M.;Schultz,D.G.
  44. Proceedings of the American Documentation Institute v.3 The Measurability of Relevance Rees,Alan M.;T.Saracevic
  45. Jr. of American Society for Information Science v.39 Performance measures for information retrieval systems-an experimental approach Regazzi,J.J.
  46. Science v.134 no.3484 Relative effectiveness of document titles and abstracts for determining the relevance of documents Resnick,A.
  47. American Documentation v.15 The consistence of human judgments of relevance Resnick,A;Savage,T.R.
  48. Proceedings of ASIS v.6 Comparative effects of titles, abstracts, and full texts on relevance judgments Saracevic,Tefko.
  49. Introduction to information science The Concept of Relevance in Information Science:A Historical Review Saracevic,Tefko;T.Saracevic(ed.)
  50. Proceedings of the American Society for Information Science v.7 Ten Years of Relevance Experimentation:A Summary and Synthesis of Conclusions Saracevic,Tefko.
  51. Jr. of American Society for Information Science v.26 Relevance:A review of and a framework for the thinking on the notion in information science Saracevic,Tefko.
  52. Jr. of American Society for Information Science v.39 A study of information seeking and retrieving. Ⅰ. Background and methology Saracevic,Tefko.(et al.)
  53. Proceeding of the ASIS Annual Meety v.28 Individual differences in organizing. searching and retrieving information Saracevic,Tefko.
  54. Information Processing and Managements v.26 no.6 A reexamination of relevance:Towards a dynamic. situational definition Schamber,Linda.;Eisenberg,Michael B.;Nilan,Michael S.
  55. Library and Information Science Research v.17 no.4 Bibliographic database searching by graduate-students in language and literature-search strategies, system interfaces, and relevance judgments Shaw,D.
  56. Information Processing and Management v.30 no.2 Information retrieval evaluation in practice:A case study approach Smithson,Steve.
  57. Relevance:Communication and cognition Sperber,D.;Wilson,D
  58. Science v.151 A metric for the social consensus Stevens,S.S.
  59. Library Quarterly v.41 no.3 Some unexplained aspects of the Cranfield Tests of indexing performance factors Swanson,Don R.
  60. Library Quarterly v.56 Subjective versus objective relevance in bibliographic retrieval systems Swanson,Don R.
  61. Jr. of American Society for Information Science v.24 The functions of abstracts in the initial screening of technical documents by the user Thompson,C.W.N.
  62. Perceptionand Psychophysics v.33 no.5 Group and individual relations between sensation magnitudes and their numerical estimation Zwsilocki,J.J.