• Title/Summary/Keyword: Distributed Information Retrieval

Search Result 168, Processing Time 0.027 seconds

PDFindexer: Distributed PDF Indexing system using MapReduce

  • Murtazaev, JAziz;Kihm, Jang-Su;Oh, Sangyoon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.4 no.1
    • /
    • pp.13-17
    • /
    • 2012
  • Indexing allows converting raw document collection into easily searchable representation. Web searching by Google or Yahoo provides subsecond response time which is made possible by efficient indexing of web-pages over the entire Web. Indexing process gets challenging when the scale gets bigger. Parallel techniques, such as MapReduce framework can assist in efficient large-scale indexing process. In this paper we propose PDFindexer, system for indexing scientific papers in PDF using MapReduce programming model. Unlike Web search engines, our target domain is scientific papers, which has pre-defined structure, such as title, abstract, sections, references. Our proposed system enables parsing scientific papers in PDF recreating their structure and performing efficient distributed indexing with MapReduce framework in a cluster of nodes. We provide the overview of the system, their components and interactions among them. We discuss some issues related with the design of the system and usage of MapReduce in parsing and indexing of large document collection.

The Design and Implementation of e-BCOS for e-Business Component System (e-비즈니스 컴포넌트 시스템 설계 및 구현)

  • Choi, Ha-Jung;Kim, Haeng-Kon
    • The KIPS Transactions:PartD
    • /
    • v.10D no.1
    • /
    • pp.85-100
    • /
    • 2003
  • Today's computing system has expanded its application to business trade and distributed work transactions using the Internet. As the demand for more flexible, adaptable, extensible, and robust web-based enterprise, these application development has been gradually expanded based on reusable, independent, and portable components. Component Based Development (CBD) works by developing and evolving software from selected reusable software components and then assembling them within appropriate software architecture. However, it requires an increase in cost to build new components as well as the necessary effort to develop of the business requirement these components. Standardized component models are required as well from the perspective of systems in order to support rapid and exact component information transmission on the web. In this paper, we describe the e-Business Component Development with agent for rapid application development on the web that correspond to the demands of users in the business domain. We design and implement the specifications of e-business components by combining these demands. In order to improve the agent register and retrieval, we propose the intelligent search and register agents, which can conduct more precise searching and specializing for components. The system enables the locating of user's frequently used components through an agent involving register and retrieval, as well as rapid procedures for registers The e-BCOS (e-Business Component System) is the agent system for the user to register distributed components and to search for components Information. The e-BCOS increases reusability through the e-business component development of distributed components in the business domain. For the share and delivery, specification with XML is acceptable to user's variable order e-BCOS Includes the effective investment, timeliness, reliability, efficiency, and maintenance effort by with agent.

Design of an Efficient Keyword-based Retrieval System Using Concept lattice (개념 망을 이용한 키워드 기반의 효율적인 정보 검색 시스템 설계)

  • Ma, Jin;Jeon, In ho;Choi, Young keun
    • Journal of Internet Computing and Services
    • /
    • v.16 no.3
    • /
    • pp.43-57
    • /
    • 2015
  • In this thesis was conducted to propose a method for efficient information retrieval using concept lattices. Since this thesis designed a new system based on ordinary concept lattices, it has the same approach method as ontology, but this thesis proposes new concept lattices to be used by establishing collaborative relations between objects and concepts that users are likely to search information more efficiently. The system suggested by this thesis can be summarized as below. Firstly, this system leads to a collaborative search by using Three kinds of concepts, such as keyword concept lattices, which focus on input key words, expert concept lattices recommended by experts and theme concept lattices, and based on these 3 concept lattices, it will help users search information they want more efficiently. Besides, as the expert concept and the keyword concept become combined, further providing users with the frequency of keyword and the frequency of category, this system can function to recommend key words related to search words entered by users. Another function of this system is to inform users of key words and categories used in users' interested themes by using the theme concept lattices. Secondly, when there is not keyword entered by a user, it is possible for users to achieve the goal of search through the secondary search when this system provides them with key words related to the input keyword. Thirdly, since most of the information is managed while being dispersed, such dispersed and managed information not only has different expression methods but changes as time goes. Accordingly, By using XMDR for efficient data access and integration of distributed information, this thesis proposes a new technique and retrieval system to integrate dispersed data.

A Study on the Design and Implementation of the System for Distributed Information Retrieval based on the Metadata using of XML (XML을 이용한 메타데이터 분산검색 시스템의 설계 및 구현에 관한 연구)

  • Song, Jong-Cheol;Hong, Gi-Chae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.11c
    • /
    • pp.2415-2418
    • /
    • 2002
  • 인터넷이 급속히 발전하고 확산되면서 정보를 효율적으로 활용하고 유통시키기 위한 연구가 활발히 진행되고 있다. OAI(Open Archives Initiative)에서는 대용량 정보를 메타데이터를 이용하여 공유하고 검색할 수 있는 프로토콜 및 임프라에 대한 연구와 표준화를 추진하고 있다. 또한 EMAF(An Extensible Multi-Agent Framework)에서는 멀티에이전트를 이용한 정보 유통 및 활용한 대한 연구가 진행중이다. 이에, 본 논문에서는 메타데이타과 XML, 멀티에이전트를 이용한 분산검색 시스템을 설계하고 구현하였다. 본 시스템은 조정에이전트와 응용에이전트로 구성되고 에이전트간 통신에는 XML과 OAI의 메타데이타 하비스팅 프로토콜을 응용하였다. 메타데이터에 대한 검색을 수행하여 검색 성능을 높일 수 있었으며 또한 사용자가 입력한 문장 단위의 질의를 처리할 수 있는 기능과 관련어를 추출할 수 있는 기능도 제공한다.

  • PDF

A Study of the Standardization Model for the National Knowledge and Information Resource (지식정보자원 표준화 모델 연구)

  • Lee, Chang-Yeol;Jeong, Eui-Suk
    • Journal of the Korean Society for information Management
    • /
    • v.23 no.4 s.62
    • /
    • pp.165-177
    • /
    • 2006
  • National Knowledge and Information Resources of KADO(Korea Agency for Digital Opportunity and Promotion) were distributed to the several data centers. The metadata for the resources was the conceptual level recommended standard. It was not for the integration, but the retrieval. So it is not easy to integrate to the central metadata DB or connect metadata among the data centers. In this paper, we analysed the metadata of the several data centers and provided the integrated standard model for the central metadata DB.

A study of intelligent Agent Using CORRA (CORBA기반 지능형 에이전트에 관한 연구)

  • 장일동;이희용
    • Journal of the Korea Society of Computer and Information
    • /
    • v.7 no.1
    • /
    • pp.88-93
    • /
    • 2002
  • In this thesis, we propose an Intelligent Agent based on CORBA technology which can exchange and share data under distributed computing environment using Agent Communication Language(ACL), Recently, due to the rapid development of network technology, the need for the database management appeared. So representative standard, CORBA have been applied and used under distributed environment. Agent applications are being developed for various fields such as electronic commerce, user interface, information retrieval. When a user searches goods, he navigates several servers on different platform until he finds desired goods. The Intelligent Agent presented in this thesis, does these tedious navigations on behalf of the user through communication between different platform.

  • PDF

Development of Integrated Retrieval System of the Biology Sequence Database Using Web Service (웹 서비스를 이용한 바이오 서열 정보 데이터베이스 및 통합 검색 시스템 개발)

  • Lee, Su-Jung;Yong, Hwan-Seung
    • The KIPS Transactions:PartD
    • /
    • v.11D no.4
    • /
    • pp.755-764
    • /
    • 2004
  • Recently, the rapid development of biotechnology brings the explosion of biological data and biological data host. Moreover, these data are highly distributed and heterogeneous, reflecting the distribution and heterogeneity of the Molecular Biology research community. As a consequence, the integration and interoperability of molecular biology databases are issue of considerable importance. But, up to now, most of the integrated systems such as link based system, data warehouse based system have many problems which are keeping the data up to date when the schema and data of the data source are changed. For this reason, the integrated system using web service technology that allow biological data to be fully exploited have been proposed. In this paper, we built the integrated system if the bio sequence information bated on the web service technology. The developed system allows users to get data with many format such as BSML, GenBank, Fasta to traverse disparate data resources. Also, it has better retrieval performance because the retrieval modules of the external database proceed in parallel.

A Distributed Domain Document Object Management using Semantic Reference Relationship (SRR을 이용한 분산 도메인 문서 객체 관리)

  • Lee, Chong-Deuk
    • Journal of Digital Convergence
    • /
    • v.10 no.5
    • /
    • pp.267-273
    • /
    • 2012
  • The semantic relationship structures hierarchically the huge amount of document objects which is usually not formatted. However, it is very difficult to structure relevant data from various distributed application domains. This paper proposed a new object management method to service the distributed domain objects by using semantic reference relationship. The proposed mechanism utilized the profile structure in order to extract the semantic similarity from application domain objects and utilized the joint matrix to decide the semantic relationship of the extracted objects. This paper performed the simulation to show the performance of the proposed method, and simulation results show that the proposed method has better retrieval performance than the existing text mining method and information extraction method.

Design of Adaptive Retrieval System using XMDR based knowledge Sharing (지식 공유 기반의 XMDR을 이용한 적응형 검색 시스템 설계)

  • Hwang Chi-Gon;Jung Kye-Dong;Choi Young-Keun
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.31 no.8B
    • /
    • pp.716-729
    • /
    • 2006
  • The information systems in the most enterprise environments are distributed locally and are comprised with various heterogeneous data sources, so that it is difficult to obtain necessary and integrated information for supporting user decision. For solving 'this problems efficiently, it provides uniform interface to users and constructed database systems between heterogeneous systems make a consistence each independence and need to provide transparency like one interface. This paper presents XMDR that consists of category, standard ontology, location ontology and knowledge base. Standard ontology solves heterogeneous problem about naming, attributes, relations in data expression. Location ontology is a mediator that connects each legacy systems. Knowledge base defines the relation for sharing glossary. Adaptive retrieve proposes integrated retrieve system through reflecting site weight by location ontology, information sharing of various forms of knowledge base and integration and propose conceptual domain model about how to share unstructured knowledge.

The Study of Distributed Bibliographic Information Retrieval System(DBIRS) Model based on CORBA and Java (CORBA/Java 기반 분산 서지정보 통합검색시스템(DBIRS) 모델에 관한 연구)

  • 서현석;박우창
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 1998.10b
    • /
    • pp.87-89
    • /
    • 1998
  • 정보 기술의 발달로 문헌정보의 양이 증가하고 있지만 정확하고 빠른 정보를 지역적으로 분산된 서지정보 데이터베이스에서 찾기에 이미 개발된 방식으로는 시간이 많이 걸린다. 현재 사용중인 텔넷(telnet)을 통한 방식이나. 웹/CGI 기술을 이용한 방식은 문헌정보가 존재하는 각각의 서버를 사용자가 한번에 하나씩 접속하여 검색을 해야 하므로 검색시간이 많이 걸릴 뿐 아니라, 사용자는 각 서버마다 상이한 검색방법을 익혀야 한다. 최근에 이러한 단점들을 개선하기 위해 한곳에서 종합적인 서지 및 본문정보검색을 하는 통합 검색시스템이 중앙 집중형 시스템을 모델로 하여 구축되고 있으나 이 방식 역시, 자료의 불일치성, 시스템 공장과 과부하등 여러 문제점들을 내포하고 있다. 따라서 본 논문에서는 이러한 문제들의 해결책으로 분산 서지정보 통합검색시스템 DBIRS를 제안하고 구현한다. DBIRS는 웹 환경하에서 CORBA 기술과 Java언어를 이용함으로서, 단일 사용자 인터페이스로 병렬 복수질의를 수행하여 동시에 각 도서관에서 자료를 찾을 수 있다. 또한 객체를 기반으로 하는 표준 모델을 사용하여 소프트웨어의 개발과 확장을 가능하게 하고, 향후 멀티미디어 데이터나 웹과 같은 비 정형 데이터 시스템과의 통합을 쉽게 해준다.