• International Journal of Internet, Broadcasting and Communication
• /
• 제4권1호
• /
• pp.13-17
• /
• 2012

Indexing allows converting raw document collection into easily searchable representation. Web searching by Google or Yahoo provides subsecond response time which is made possible by efficient indexing of web-pages over the entire Web. Indexing process gets challenging when the scale gets bigger. Parallel techniques, such as MapReduce framework can assist in efficient large-scale indexing process. In this paper we propose PDFindexer, system for indexing scientific papers in PDF using MapReduce programming model. Unlike Web search engines, our target domain is scientific papers, which has pre-defined structure, such as title, abstract, sections, references. Our proposed system enables parsing scientific papers in PDF recreating their structure and performing efficient distributed indexing with MapReduce framework in a cluster of nodes. We provide the overview of the system, their components and interactions among them. We discuss some issues related with the design of the system and usage of MapReduce in parsing and indexing of large document collection.

• 한국도서관정보학회지
• /
• 제10권
• /
• pp.95-131
• /
• 1983

Index is a systematic guide to items contained in, or concepts derived from, a collection, Thus, it is represented as a paired set of index terms (t) and documents (D) : I= {(t,D) vertical bar t .mem. V, D .mem. W), where V is index vocabulary and W is document collection. Indexing is the process of analysing the informational content of records of knowledge and expressing the informational content in the language of the indexing system. It involves: 1) Selecting indexable concepts in a document; and 2) expressing these concepts in the language of the indexing system (as index entries): and an ordered list. Indexing process involves technical, semantic and syntactic problems. Technical problems are related to the accuracy of indexing, which is primarily governed by the indexer's ability of analysing subject, identifying indexable concepts, and coding. The proper levels of indexing exhaustivity, and index language specificity are also significant factors affecting the quality of index. Semantic problems are related to the choice of index terms and the form in which they should be used. Equivalent, hierarchical and affinitive/associative relationships of index terms are involved. Syntactic problems are largely related to the coordination of index terms. This process of coordination arises from the need to be able to search for the intersection of two or more classes defined by terms denoting distinct concepts. Finally, most valuable aspects of alphabetical subject indexing theories and practices are derived from those of Cutter, Kaiser, Ranganathan, Coates, Lynch and Austin, and discussed in details.

• 한국정보처리학회논문지
• /
• 제6권2호
• /
• pp.315-325
• /
• 1999

Indexing techniques for distributed resources have much effect on an information service system based on distributed environment like digital library. There is a centralized indexing technique, a distributed technique, and a mixed technique for distributed indexing techniques. In this paper, we propose new distributed indexing technique using EIF(extended Inverted File) structure that mix the centralized technique and t도 distributed technique. And we propose management techniques using EIF structure and retrieval technique using EIF structure. This distributed indexing technique proposed is able to fast process retrieval request and reduce network overload and select servers relevant to query terms. This paper investigated performance of a proposed distributed indexing technique.

• 한국지능정보시스템학회:학술대회논문집
• /
• 한국지능정보시스템학회 2001년도 The Pacific Aisan Confrence On Intelligent Systems 2001
• /
• pp.237-245
• /
• 2001

Automated essay grading has been proposed for over thirty years. Only recently have practical implementations been constructed and tested. This paper investigated the role of the nearest-neighbour algorithm within the information retrieval as a way of grading the essay automatically called Automated Essay Grading System. It intended to offer teachers an individualized assistance in grading the student\`s essay. The system involved several processes, which are the indexing, the structuring of the model answer and the grade processing. The indexing process comprised the document indexing and query processing which are mainly used for representing the documents and the query. Structuring the model answer is actually preparing the marking scheme and the grade processing is the process of assessing the essay. To test the effectiveness of the developed algorithms, the algorithms are tested against the History text in Malay. The result showed that th information retrieval and the nearest-neighbour algorithm are practical combination that offer acceptable performance for grading the essay.

• 정보처리학회논문지D
• /
• 제13D권3호
• /
• pp.317-324
• /
• 2006

본 논문에서는 다양한 질의 처리를 위하여 데이터베이스 색인 스키마의 특징인 반구조적 정보를 구조 정보화할 수 있는 방법과 보다 빠르고 최적화된 질의처리 방법을 제안하고자 한다. XML 트리에 추가된 번호 정보를 비트열로 변형하여 트리의 구조 변경없이 트리 노드간의 구조 정보를 나타낸다. 그리고 이 과정에서 생성되는 기타 구조 정보들을 검색하여 색인 스키마에 추가한다. 또한, 질의 스키마에서는 색인 스키마를 이용하여 절대 경로 질의 표현식 뿐만 아니라 상대 경로 질의 표현식의 경우에도 주어진 노드 정보를 통하여 상위의 노드를 복원한다. 이러한 점은 하나의 질의를 통하여 파생 질의 표현식을 작성할 수 있다는 장점이 있다. 그리고 질의 처리 과정에서는 색인 스키마와 질의 스키마를 이용하여 비트열 사이의 비트 연산을 함으로써 응답시간을 최소화하고 색인 파일의 노드별 레코드의 정보만으로 정확한 결과를 검색할 수 있다.

• 한국도서관정보학회지
• /
• 제12권
• /
• pp.61-96
• /
• 1985

The human subject analysis of a document has some critical problems. The method results in the inconsistency in analysis process and the contradiction of two objects of the subject analysis (one is the identification of the content for the retrieval of specific items and the other is to identify the content for the grouping of related materials). Since the subject analysis by mechanized has been recognized to be the possible way to aggregate the problems of manual analysis, various a n.0, pproaches of automatic indexing have been studied and experimented. This study is to examine the automatic indexing as one of the promising subject analysis techniques by statistical, syntactical and semantic a n.0, pproaches. In conclusion, the reasonable a n.0, pplication time of the automatic indexing should be made a decision based on the through investigation on the cost verse effectiveness, and automatic indexing system should be developed in the close relationship with the on-line search which is a good retrieval system for information explosion society. From now on, since the machine-readable document-text will be envisaged to be more and more available due to the rapid development of computer technology, the more substantial research on the automatic indexing will be also possible, which can bring about the increasing of practical automatic indexing systems.

• Journal of Computing Science and Engineering
• /
• 제6권2호
• /
• pp.151-160
• /
• 2012

We present a methodology that automatically selects indexing algorithms for each heading in Medical Subject Headings (MeSH), National Library of Medicine's vocabulary for indexing MEDLINE. While manually comparing indexing methods is manageable with a limited number of MeSH headings, a large number of them make automation of this selection desirable. Results show that this process can be automated, based on previously indexed MEDLINE citations. We find that AdaBoostM1 is better suited to index a group of MeSH hedings named Check Tags, and helps improve the micro F-measure from 0.5385 to 0.7157, and the macro F-measure from 0.4123 to 0.5387 (both p < 0.01).

• ETRI Journal
• /
• 제29권5호
• /
• pp.691-693
• /
• 2007

Existing methods to process continuous range queries are not scalable. In particular, as the number of continuous range queries on a large number of moving objects becomes larger, their performance degrades significantly. We propose a novel query indexing method called the projected attribute bit (PAB)-based query index. We project a two-dimensional continuous range query on each axis to get two one-dimensional bit lists. Since the queries are transformed to bit lists and query evaluation is performed by bit operations, the storage cost of indexing and query evaluation time are reduced significantly. Through various experiments, we show that our method outperforms the containment-encoded squares-based indexing method, which is one of the most recently proposed methods.

• 한국문헌정보학회지
• /
• 제25권
• /
• pp.51-82
• /
• 1993

Effective indexing systems are required to enhance the performance of full-text retrieval systems. The result of the analysis of index terms selected by human indexers without a newspaper thesaurus indicates that controlled indexing language is necessary for effective and consistent indexing of newspaper articles. In this paper, basic principles are established for keyword selection from Korean newspapers and significant problems identified in the process of developing a newspaper thesaurus are discussed in depth.

• 제어로봇시스템학회:학술대회논문집
• /
• 제어로봇시스템학회 2004년도 ICCAS
• /
• pp.810-813
• /
• 2004

Indexing and Retrieving images from large and varied collections using image content as a key is a challenging and important problem in computer vision application. In this paper, a color Content-based Image Retrieval (CBIR) system using hierarchical Region of Interest (ROI) query and indexing is presented. During indexing process, First, The ROIs on every image in the image database are extracted using a region-based image segmentation technique, The JSEG approach is selected to handle this problem in order to create color-texture regions. Then, Color features in form of histogram and correlogram are then extracted from each segmented regions. Finally, The features are stored in the database as the key to retrieve the relevant images. As in the retrieval system, users are allowed to select ROI directly over the sample or user's submission image and the query process then focuses on the content of the selected ROI in order to find those images containing similar regions from the database. The hierarchical region-of-interest query is performed to retrieve the similar images. Two-level search is exploited in this paper. In the first level, the most important regions, usually the large regions at the center of user's query, are used to retrieve images having similar regions using static search. This ensures that we can retrieve all the images having the most important regions. In the second level, all the remaining regions in user's query are used to search from all the retrieved images obtained from the first level. The experimental results using the indexing technique show good retrieval performance over a variety of image collections, also great reduction in the amount of searching time.