• Title/Summary/Keyword: Retrieval Evaluation

Search Result 345, Processing Time 0.024 seconds

Implementation and Verification of Dynamic Search Ranking Model for Information Search Tasks: The Evaluation of Users' Relevance Judgement Model (정보 검색 과제별 동적 검색 랭킹 모델 구현 및 검증: 사용자 중심 적합성 판단 모형 평가를 중심으로)

  • Park, Jung-Ah;Sohn, Young-Woo
    • Science of Emotion and Sensibility
    • /
    • v.15 no.3
    • /
    • pp.367-380
    • /
    • 2012
  • The purpose of this research was to implement and verify an information retrieval(IR) system based on users' relevance criteria for information search tasks. For this purpose, we implemented an IR system with a dynamic ranking model using users' relevance criteria varying with the types of information search task and evaluated this system through user experiment. 45 participants performed three information search tasks on both IR systems with a static and a dynamic ranking model. Three Information search tasks are fact finding search task, problem solving search task and decision making search task. Participants evaluated top five search results on 7 likert scales of relevance. We observed that the IR system with a dynamic ranking model provided more relevant search results compared to the system with a static ranking model. This research has significance in designing IR system for information search tasks, in testing the validity of user-oriented relevance judgement model by implementing an IR system for actual information search tasks and in relating user research to the improvement of an IR system.

  • PDF

Detecting Errors in POS-Tagged Corpus on XGBoost and Cross Validation (XGBoost와 교차검증을 이용한 품사부착말뭉치에서의 오류 탐지)

  • Choi, Min-Seok;Kim, Chang-Hyun;Park, Ho-Min;Cheon, Min-Ah;Yoon, Ho;Namgoong, Young;Kim, Jae-Kyun;Kim, Jae-Hoon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.9 no.7
    • /
    • pp.221-228
    • /
    • 2020
  • Part-of-Speech (POS) tagged corpus is a collection of electronic text in which each word is annotated with a tag as the corresponding POS and is widely used for various training data for natural language processing. The training data generally assumes that there are no errors, but in reality they include various types of errors, which cause performance degradation of systems trained using the data. To alleviate this problem, we propose a novel method for detecting errors in the existing POS tagged corpus using the classifier of XGBoost and cross-validation as evaluation techniques. We first train a classifier of a POS tagger using the POS-tagged corpus with some errors and then detect errors from the POS-tagged corpus using cross-validation, but the classifier cannot detect errors because there is no training data for detecting POS tagged errors. We thus detect errors by comparing the outputs (probabilities of POS) of the classifier, adjusting hyperparameters. The hyperparameters is estimated by a small scale error-tagged corpus, in which text is sampled from a POS-tagged corpus and which is marked up POS errors by experts. In this paper, we use recall and precision as evaluation metrics which are widely used in information retrieval. We have shown that the proposed method is valid by comparing two distributions of the sample (the error-tagged corpus) and the population (the POS-tagged corpus) because all detected errors cannot be checked. In the near future, we will apply the proposed method to a dependency tree-tagged corpus and a semantic role tagged corpus.

A Study on the National Teacher Recruiting Examination for School Librarian Teacher: Focusing on the School Library Practice Area (사서교사 임용시험 출제경향 고찰 - 학교도서관 실무영역을 중심으로 -)

  • Kyungkuk Noh;Jeonghoon Lim
    • Journal of Korean Library and Information Science Society
    • /
    • v.54 no.4
    • /
    • pp.85-104
    • /
    • 2023
  • The purpose of this study is to analyze the examination questions used in the librarian teacher recruitment exam, including the domains, content, and evaluation factors, and to propose improvements for the recruitment exam. To achieve this, examination questions for librarian teacher recruitment exams since 2002, provided by the Korea Institute for Curriculum and Evaluation, were collected and analyzed the frequency of appearances by section. The analysis revealed that, 106 questions (21.95%) on school library administration, 63 questions (13.04%) on classification and information retrieval 59 questions (12.22%) on library computerization, 58 questions (12.01%) on reading education, 56 questions (11.59%) cataloging and information service, and 18 questions (3.73%) on information media were examined. Next, analyzed the frequency of appearances in the last 10 years (2014-2023) by dividing the examination areas into specialty of librarian and school library practice, and found that there were a total of 149 questions (66.22%) related to specialty of librarian and 76 questions (33.78%) related to school library practice. Based on these findings, recommendations have been made for update assessment areas and factors, expanding the field of information media, and suggested the need for a stable and continuous teacher recruitment policy.

A Study of Performance Analysis on Effective Multiple Buffering and Packetizing Method of Multimedia Data for User-Demand Oriented RTSP Based Transmissions Between the PoC Box and a Terminal (PoC Box 단말의 RTSP 운용을 위한 사용자 요구 중심의 효율적인 다중 수신 버퍼링 기법 및 패킷화 방법에 대한 성능 분석에 관한 연구)

  • Bang, Ji-Woong;Kim, Dae-Won
    • Journal of Korea Multimedia Society
    • /
    • v.14 no.1
    • /
    • pp.54-75
    • /
    • 2011
  • PoC(Push-to-talk Over Cellular) is an integrated technology of group voice calls, video calls and internet based multimedia services. If a PoC user can not participate in the PoC session for various reasons such as an emergency situation, lack of battery capacity, then the user can use the PoC Box which has a similar functionality to the MM Box in the MMS(Multimedia Messaging Service). The RTSP(Real-Time Streaming Protocol) method is recommended to be used when there is a transmission session between the PoC box and a terminal. Since the existing VOD service uses a wired network, the packet size of RTSP-based VOD service is huge, however, the PoC service has wireless communication environments which have general characteristics to be used in RTSP method. Packet loss in a wired communication environments is relatively less than that in wireless communication environment, therefore, a buffering latency occurs in PoC service due to a play-out delay which means an asynchronous play of audio & video contents. Those problems make a user to be difficult to find the information they want when the media contents are played-out. In this paper, the following techniques and methods were proposed and their performance and superiority were verified through testing: cross-over dual reception buffering technique, advance partition multi-reception buffering technique, and on-demand multi-reception buffering technique, which are designed for effective picking up of information in media content being transmitted in short amount of time using RTSP when a user searches for media, as well as for reduction in playback delay; and same-priority packetization transmission method and priority-based packetization transmission method, which are media data packetization methods for transmission. From the simulation of functional evaluation, we could find that the proposed multiple receiving buffering and packetizing methods are superior, with respect to the media retrieval inclination, to the existing single receiving buffering method by 6-9 points from the viewpoint of effectiveness and excellence. Among them, especially, on-demand multiple receiving buffering technology with same-priority packetization transmission method is able to manage the media search inclination promptly to the requests of users by showing superiority of 3-24 points above compared to other combination methods. In addition, users could find the information they want much quickly since large amount of informations are received in a focused media retrieval period within a short time.

Evaluation of the Satellite-based Air Temperature for All Sky Conditions Using the Automated Mountain Meteorology Station (AMOS) Records: Gangwon Province Case Study (산악기상관측정보를 이용한 위성정보 기반의 전천후 기온 자료의 평가 - 강원권역을 중심으로)

  • Jang, Keunchang;Won, Myoungsoo;Yoon, Sukhee
    • Korean Journal of Agricultural and Forest Meteorology
    • /
    • v.19 no.1
    • /
    • pp.19-26
    • /
    • 2017
  • Surface air temperature ($T_{air}$) is a key variable for the meteorology and climatology, and is a fundamental factor of the terrestrial ecosystem functions. Satellite remote sensing from the Moderate Resolution Imaging Spectroradiometer (MODIS) provides an opportunity to monitor the $T_{air}$. However, the several problems such as frequent cloud cover and mountainous region can result in substantial retrieval error and signal loss in MODIS $T_{air}$. In this study, satellite-based $T_{air}$ was estimated under both clear and cloudy sky conditions in Gangwon Province using Aqua MODIS07 temperature profile product (MYD07_L2) and GCOM-W1 Advanced Microwave Scanning Radiometer 2 (AMSR2) brightness temperature ($T_b$) at 37 GHz frequency, and was compared with the measurements from the Automated Mountain Meteorology Stations (AMOS). The application of ambient temperature lapse rate was performed to improve the retrieval accuracy in mountainous region, which showed the improvement of estimation accuracy approximately 4% of RMSE. A simple pixel-wise regression method combining synergetic information from MYD07_L2 $T_{air}$ and AMSR2 $T_b$ was applied to estimate surface $T_{air}$ for all sky conditions. The $T_{air}$ retrievals showed favorable agreement in comparison with AMOS data (r=0.80, RMSE=7.9K), though the underestimation was appeared in winter season. Substantial $T_{air}$ retrievals were estimated 61.4% (n=2,657) for cloudy sky conditions. The results presented in this study indicate that the satellite remote sensing can produce the surface $T_{air}$ at the complex mountainous region for all sky conditions.

A Study on the Research Trends in Library & Information Science in Korea using Topic Modeling (토픽모델링을 활용한 국내 문헌정보학 연구동향 분석)

  • Park, Ja-Hyun;Song, Min
    • Journal of the Korean Society for information Management
    • /
    • v.30 no.1
    • /
    • pp.7-32
    • /
    • 2013
  • The goal of the present study is to identify the topic trend in the field of library and information science in Korea. To this end, we collected titles and s of the papers published in four major journals such as Journal of the Korean Society for information Management, Journal of the Korean Society for Library and Information Science, Journal of Korean Library and Information Science Society, and Journal of the Korean BIBLIA Society for library and Information Science during 1970 and 2012. After that, we applied the well-received topic modeling technique, Latent Dirichlet Allocation(LDA), to the collected data sets. The research findings of the study are as follows: 1) Comparison of the extracted topics by LDA with the subject headings of library and information science shows that there are several distinct sub-research domains strongly tied with the field. Those include library and society in the domain of "introduction to library and information science," professionalism, library and information policy in the domain of "library system," library evaluation in the domain of "library management," collection development and management, information service in the domain of "library service," services by library type, user training/information literacy, service evaluation, classification/cataloging/meta-data in the domain of "document organization," bibliometrics/digital libraries/user study/internet/expert system/information retrieval/information system in the domain of "information science," antique documents in the domain of "bibliography," books/publications in the domain of "publication," and archival study. The results indicate that among these sub-domains, information science and library services are two most focused domains. Second, we observe that there is the growing trend in the research topics such as service and evaluation by library type, internet, and meta-data, but the research topics such as book, classification, and cataloging reveal the declining trend. Third, analysis by journal show that in Journal of the Korean Society for information Management, information science related topics appear more frequently than library science related topics whereas library science related topics are more popular in the other three journals studied in this paper.

A Study on the Indexing System Using a Controlled Vocabulary and Natural Language in the Secondary Legal Information Full-Text Databases : an Evaluation and Comparison of Retrieval Effectiveness (2차 법률정보 전문데이터베이스에 있어서 통제어 색인시스템과 자연어 색인시스템의 검색효율 평가에 관한 연구)

  • Roh Jeong-Ran
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.32 no.4
    • /
    • pp.69-86
    • /
    • 1998
  • The purpose of velop the indexing algorithm of secondary legal information by the study of characteristics of legal information, to compare the indexing system using controlled vocabulary to the indexing system using natural language in the secondary legal information full-text databases, and to prove propriety and superiority of the indexing system using controlled vocabulary. The results are as follows; 1)The indexing system using controlled vocabulary in the secondary legal information full-text databases has more effectiveness than the indexing system using natural language, in the recall rate, the precision rate, the distribution of propriety, and the faculty of searching for the unique proper-records which the indexing system using natural language fans to find 2)The indexing system which adds more words to the controlled vocabulary in the secondary legal information full-text databases does not better effectiveness in the retail rate, the precision rate, comparing to the indexing system using controlled vocabulary. 3)The indexing system using word-added controlled vocabulary with an extra weight in the secondary legal information full-text databases does not better effectiveness in the recall rate, the precision rate, comparing to the indexing system using word-added controlled vocabulary without an extra weight. This study indicates that it is necessary to have characteristic information the information experts recognize - that is to say, experimental and inherent knowledge only human being can have built-in into the system rather than to approach the information system by the linguistic, statistic or structuralistic way, and it can be more essential and intelligent information system.

  • PDF

Information Architecture Design Using Eye-tracking Method (Eye-Tracking Method를 이용한 메뉴구조 설계 및 평가)

  • Park, Jong-Soon;Myung, Ro-Hae
    • Journal of the HCI Society of Korea
    • /
    • v.2 no.1
    • /
    • pp.33-39
    • /
    • 2007
  • Because of the cognitive overload which is caused by the complicated information structure, Digital Convergence product interferes with the effective retrieval of the information from the menu. Two methods have been used to alleviate that cognitive overload by making an effective menu structure; physical menu structure method which is related with the width and depth of the menu, semantic menu structure method which is related with the menu title. In this research, we tried to demonstrate the effectiveness of the menu structure designing method by suggesting a new semantic methodology which uses the Fixation and Fixation duration which are accompanied by the visual search. Because the Fixation is automatically processed by the human cognitive model, we could easily recognize whether the information structure is correspond to the cognitive model or not. From this fact we established the hypothesis that the number of cognitively well established menu structures are fewer than that of the wrongly designed menu structures in terms of the Fixation number and Duration. To verify this hypothesis, we compared the Fixation number and Duration of the modified menu structures with those of the original menu structures by using the Eye-Tracking experiment. As a result, we could find the significant decrease of the Fixation number and Duration after modification. Therefore we could recognize that the modified menu structure was more effective than the original menu structure. In sum, the newly suggested menu structure designing methodology which uses the Fixation and Fixation Duration accompanied by the visual search was proved to be a very effective method.

  • PDF

Inpatient Dental Consultations to Pediatric Dentistry in the Yonsei University Severance Hospital (연세대학교 세브란스 병원 내 입원한 환자의 소아치과 의뢰 현황)

  • Joo, Kihoon;Lee, Jaeho;Song, Jeseon;Lee, Hyoseol
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • v.41 no.2
    • /
    • pp.145-151
    • /
    • 2014
  • The goal of this study was to describe dental consultation of pediatric inpatients to the department of pediatric dentistry at Yonsei University Severance Hospital. 391 dental consultations at Yonsei University Severance Hospital referred to pediatric dentistry in the year 2012 were included in this study. Consultations were categorized according to patients' gender, age, chief complaint, referred department and diagnosis. 288 patients (166 males and 122 females) with an average age of 5.9 were referred to the Department of Pediatric Dentistry. 129 cases (33.1%) from Department of Rehabilitation Medicine, 80 cases (20.5%) from Pediatric Hematology- Oncology, 51 cases (13.0%) from Pediatric Cardiology, and 44 cases (11.3%) from Pediatric Neurology. Chief complaints were ranked from oral examination (39.7%), dental caries (14.0%), pre-operative evaluation (12.8%) and others (33.5%); including oral pain, trauma, tooth mobility, orthodontic treatment, self-injury, fabrication of obturator and etc. Dental consultations should be encouraged as dental care and treatment could affect the control of systemic diseases of admitted patients. Pediatric inpatients have been referred to pediatric dentistry for not only comprehensive oral exam but also various chief complaints. The most frequent dental diagnosis made and treatment performed were dental caries and non-invasive/preventive care respectively.

Detecting Research Trends in Korean Information Science Research, 2000-2011 (국내 정보학분야 연구동향 분석, 2000-2011)

  • Seo, Eun-Gyoung;Yu, So-Young
    • Journal of the Korean Society for information Management
    • /
    • v.30 no.4
    • /
    • pp.215-239
    • /
    • 2013
  • Even though the overall scholarly community has recognized a dramatic growth and changes in the Information Science research in Korea over the last few decades, there are still only few studies that have identified the changes in terms of long-term and dynamic point of view. We have analyzed 1,007 IS-research articles from leading Korean journals in KCI (Korea Citation Index), published between 2000 and 2011. To discern the trendline of changes in research interests over time, we conducted a time-series analysis by developing grounded subject scheme from the article set and checking the growth rate of the number of published articles and title keywords. A comparative analysis was also conducted by constructing and comparing co-word maps over time to discover visible changes in research topics over this 12-year period of the IS-research in Korea. As a result, we identified some developments and transformations in major subject areas and knowledge structure of the IS-research in Korea over time. The major trend we discovered is that IS-studies over the 12-year period evolved from system-oriented research to library-application research. The changes are especially observed in knowledge management, Web-based system evaluation, and information retrieval areas. When compared to the results of other studies, the result of our study may serve as an evidence of the localization of Korean IS-studies in the first decade of the $21^{st}$ century.