• International Journal of Internet, Broadcasting and Communication
• /
• 제4권1호
• /
• pp.13-17
• /
• 2012

Indexing allows converting raw document collection into easily searchable representation. Web searching by Google or Yahoo provides subsecond response time which is made possible by efficient indexing of web-pages over the entire Web. Indexing process gets challenging when the scale gets bigger. Parallel techniques, such as MapReduce framework can assist in efficient large-scale indexing process. In this paper we propose PDFindexer, system for indexing scientific papers in PDF using MapReduce programming model. Unlike Web search engines, our target domain is scientific papers, which has pre-defined structure, such as title, abstract, sections, references. Our proposed system enables parsing scientific papers in PDF recreating their structure and performing efficient distributed indexing with MapReduce framework in a cluster of nodes. We provide the overview of the system, their components and interactions among them. We discuss some issues related with the design of the system and usage of MapReduce in parsing and indexing of large document collection.

• Journal of Korean Library and Information Science Society
• /
• 제10권
• /
• pp.95-131
• /
• 1983

Index is a systematic guide to items contained in, or concepts derived from, a collection, Thus, it is represented as a paired set of index terms (t) and documents (D) : I= {(t,D) vertical bar t .mem. V, D .mem. W), where V is index vocabulary and W is document collection. Indexing is the process of analysing the informational content of records of knowledge and expressing the informational content in the language of the indexing system. It involves: 1) Selecting indexable concepts in a document; and 2) expressing these concepts in the language of the indexing system (as index entries): and an ordered list. Indexing process involves technical, semantic and syntactic problems. Technical problems are related to the accuracy of indexing, which is primarily governed by the indexer's ability of analysing subject, identifying indexable concepts, and coding. The proper levels of indexing exhaustivity, and index language specificity are also significant factors affecting the quality of index. Semantic problems are related to the choice of index terms and the form in which they should be used. Equivalent, hierarchical and affinitive/associative relationships of index terms are involved. Syntactic problems are largely related to the coordination of index terms. This process of coordination arises from the need to be able to search for the intersection of two or more classes defined by terms denoting distinct concepts. Finally, most valuable aspects of alphabetical subject indexing theories and practices are derived from those of Cutter, Kaiser, Ranganathan, Coates, Lynch and Austin, and discussed in details.

• The Transactions of the Korea Information Processing Society
• /
• 제6권2호
• /
• pp.315-325
• /
• 1999

Indexing techniques for distributed resources have much effect on an information service system based on distributed environment like digital library. There is a centralized indexing technique, a distributed technique, and a mixed technique for distributed indexing techniques. In this paper, we propose new distributed indexing technique using EIF(extended Inverted File) structure that mix the centralized technique and t도 distributed technique. And we propose management techniques using EIF structure and retrieval technique using EIF structure. This distributed indexing technique proposed is able to fast process retrieval request and reduce network overload and select servers relevant to query terms. This paper investigated performance of a proposed distributed indexing technique.

• Proceedings of the Korea Inteligent Information System Society Conference
• /
• 한국지능정보시스템학회 2001년도 The Pacific Aisan Confrence On Intelligent Systems 2001
• /
• pp.237-245
• /
• 2001

Automated essay grading has been proposed for over thirty years. Only recently have practical implementations been constructed and tested. This paper investigated the role of the nearest-neighbour algorithm within the information retrieval as a way of grading the essay automatically called Automated Essay Grading System. It intended to offer teachers an individualized assistance in grading the student\`s essay. The system involved several processes, which are the indexing, the structuring of the model answer and the grade processing. The indexing process comprised the document indexing and query processing which are mainly used for representing the documents and the query. Structuring the model answer is actually preparing the marking scheme and the grade processing is the process of assessing the essay. To test the effectiveness of the developed algorithms, the algorithms are tested against the History text in Malay. The result showed that th information retrieval and the nearest-neighbour algorithm are practical combination that offer acceptable performance for grading the essay.

• The KIPS Transactions:PartD
• /
• 제13D권3호
• /
• pp.317-324
• /
• 2006

In this paper, we propose a new indexing technique to solve various queries which have a strong good point not only database indexing schema take advantage of converting from semi-structured data to structured data but also performance is more faster than before. We represent structure information of XML document between nodes of tree that additional numbering information which can be bit-stream without modified structure of XML tree. And, We add in indexing schema searching incidental structure information in the process. In Querying schema, we recover ancestor nodes through give information of node using indexing schema in complete path query expression as well as relative path query expression. Therefore, it takes advantage of making derivative query expression with given query. In this process, we recognize that indexing and querying schema can get searched result set faster and more accurate. Because response time is become shorter by bit operating, when query occur and it just needs information of record set earch node in database.

• Journal of Korean Library and Information Science Society
• /
• 제12권
• /
• pp.61-96
• /
• 1985

The human subject analysis of a document has some critical problems. The method results in the inconsistency in analysis process and the contradiction of two objects of the subject analysis (one is the identification of the content for the retrieval of specific items and the other is to identify the content for the grouping of related materials). Since the subject analysis by mechanized has been recognized to be the possible way to aggregate the problems of manual analysis, various a n.0, pproaches of automatic indexing have been studied and experimented. This study is to examine the automatic indexing as one of the promising subject analysis techniques by statistical, syntactical and semantic a n.0, pproaches. In conclusion, the reasonable a n.0, pplication time of the automatic indexing should be made a decision based on the through investigation on the cost verse effectiveness, and automatic indexing system should be developed in the close relationship with the on-line search which is a good retrieval system for information explosion society. From now on, since the machine-readable document-text will be envisaged to be more and more available due to the rapid development of computer technology, the more substantial research on the automatic indexing will be also possible, which can bring about the increasing of practical automatic indexing systems.

• Journal of Computing Science and Engineering
• /
• 제6권2호
• /
• pp.151-160
• /
• 2012

We present a methodology that automatically selects indexing algorithms for each heading in Medical Subject Headings (MeSH), National Library of Medicine's vocabulary for indexing MEDLINE. While manually comparing indexing methods is manageable with a limited number of MeSH headings, a large number of them make automation of this selection desirable. Results show that this process can be automated, based on previously indexed MEDLINE citations. We find that AdaBoostM1 is better suited to index a group of MeSH hedings named Check Tags, and helps improve the micro F-measure from 0.5385 to 0.7157, and the macro F-measure from 0.4123 to 0.5387 (both p < 0.01).

• ETRI Journal
• /
• 제29권5호
• /
• pp.691-693
• /
• 2007

Existing methods to process continuous range queries are not scalable. In particular, as the number of continuous range queries on a large number of moving objects becomes larger, their performance degrades significantly. We propose a novel query indexing method called the projected attribute bit (PAB)-based query index. We project a two-dimensional continuous range query on each axis to get two one-dimensional bit lists. Since the queries are transformed to bit lists and query evaluation is performed by bit operations, the storage cost of indexing and query evaluation time are reduced significantly. Through various experiments, we show that our method outperforms the containment-encoded squares-based indexing method, which is one of the most recently proposed methods.

• Journal of the Korean Society for Library and Information Science
• /
• 제25권
• /
• pp.51-82
• /
• 1993

Effective indexing systems are required to enhance the performance of full-text retrieval systems. The result of the analysis of index terms selected by human indexers without a newspaper thesaurus indicates that controlled indexing language is necessary for effective and consistent indexing of newspaper articles. In this paper, basic principles are established for keyword selection from Korean newspapers and significant problems identified in the process of developing a newspaper thesaurus are discussed in depth.

• 제어로봇시스템학회:학술대회논문집
• /
• 제어로봇시스템학회 2004년도 ICCAS
• /
• pp.810-813
• /
• 2004

Indexing and Retrieving images from large and varied collections using image content as a key is a challenging and important problem in computer vision application. In this paper, a color Content-based Image Retrieval (CBIR) system using hierarchical Region of Interest (ROI) query and indexing is presented. During indexing process, First, The ROIs on every image in the image database are extracted using a region-based image segmentation technique, The JSEG approach is selected to handle this problem in order to create color-texture regions. Then, Color features in form of histogram and correlogram are then extracted from each segmented regions. Finally, The features are stored in the database as the key to retrieve the relevant images. As in the retrieval system, users are allowed to select ROI directly over the sample or user's submission image and the query process then focuses on the content of the selected ROI in order to find those images containing similar regions from the database. The hierarchical region-of-interest query is performed to retrieve the similar images. Two-level search is exploited in this paper. In the first level, the most important regions, usually the large regions at the center of user's query, are used to retrieve images having similar regions using static search. This ensures that we can retrieve all the images having the most important regions. In the second level, all the remaining regions in user's query are used to search from all the retrieved images obtained from the first level. The experimental results using the indexing technique show good retrieval performance over a variety of image collections, also great reduction in the amount of searching time.