• 제목/요약/키워드: Text-search

검색결과 546건 처리시간 0.032초

Improving Elasticsearch for Chinese, Japanese, and Korean Text Search through Language Detector

  • Kim, Ki-Ju;Cho, Young-Bok
    • Journal of information and communication convergence engineering
    • /
    • 제18권1호
    • /
    • pp.33-38
    • /
    • 2020
  • Elasticsearch is an open source search and analytics engine that can search petabytes of data in near real time. It is designed as a distributed system horizontally scalable and highly available. It provides RESTful APIs, thereby making it programming-language agnostic. Full text search of multilingual text requires language-specific analyzers and field mappings appropriate for indexing and searching multilingual text. Additionally, a language detector can be used in conjunction with the analyzers to improve the multilingual text search. Elasticsearch provides more than 40 language analysis plugins that can process text and extract language-specific tokens and language detector plugins that can determine the language of the given text. This study investigates three different approaches to index and search Chinese, Japanese, and Korean (CJK) text (single analyzer, multi-fields, and language detector-based), and identifies the advantages of the language detector-based approach compared to the other two.

상용 학술데이터베이스의 텍스트 기반 검색과 비주얼검색의 사용성에 관한 연구 (Usability Evaluation of Text-based Search and Visual Search of a Multidisciplinary Library Database)

  • 김종애
    • 정보관리학회지
    • /
    • 제26권3호
    • /
    • pp.111-129
    • /
    • 2009
  • 본 연구는 시각화 정보검색시스템이 실제 정보검색환경에서 이용자에게 원활하게 수용될 수 있는지에 대한 경험적인 분석을 제공하고자, 상용 학술데이터베이스의 텍스트 기반 검색과 비주얼검색의 사용성을 비교 평가하고, 실험순서에 따라 사용성 평가에 있어 차이가 있는지 분석하였다. 검색소요시간과 처리동작횟수에 있어서 텍스트 기반 검색이 비주얼검색보다 더 효율적인 것으로 나타났으며, 통계적으로 유의한 차이가 있는 것으로 나타났다. 또한 사용성에 대한 인식에 있어서도 텍스트 기반 검색이 비주얼 검색보다 전체적으로 더 높게 나타났으며 통계적으로 유의한 차이가 있는 것으로 나타났다.

퍼지 논리를 이용한 사용자 중심적인 Full-Text 검색방법에 관한 연구 (Consideration of a Robust Search Methodology that could be used in Full-Text Information Retrieval Systems)

  • 이원부
    • Asia pacific journal of information systems
    • /
    • 제1권1호
    • /
    • pp.87-101
    • /
    • 1991
  • The primary purpose of this study was to investigate a robust search methodology that could be used in full-text information retrieval systems. A robust search methodology is one that can be easily used by a variety of users (particularly naive users) and it will give them comparable search performance regardless of their different expertise or interests In order to develop a possibly robust search methodology, a fully functional prototype of a fuzzy knowledge based information retrieval system was developed. Also, an experiment that used this prototype information retreival system was designed to investigate the performance of that search methodology over a small exploratory sample of user queries To probe the relatonships between the possibly robust search performance and the query organization using fuzzy inference logic, the search performance of a shallow query structure was analyzes. Consequently the following several noteworthy findings were obtained: 1) the hierachical(tree type) query structure might be a better query organization than the linear type query structure 2) comparing with the complex tree query structure, the simple tree query structure that has at most three levels of query might provide better search performance 3) the fuzzy search methodology that employs a proper levels of cut-off value might provide more efficient search performance than the boolean search methodology. Even though findings could not be statistically verified because the experiments were done using a single replication, it is worth noting however, that the research findings provided valuable information for developing a possibly robust search methodology in full-text information retrieval.

  • PDF

전문에 대한 검색시스템의 구현 (Implementation of Information Retrieval System for Full-Text)

  • 김대규;정희택;강영만;한순희;조혁현
    • 한국정보통신학회:학술대회논문집
    • /
    • 한국해양정보통신학회 2000년도 추계종합학술대회
    • /
    • pp.337-340
    • /
    • 2000
  • 인터넷을 이용한 정보검색이 일반화되면서, 보다 정확하고 꼭 필요한 정보의 요구가 일반화되었다. 정확한 정보의 제공을 위해, 된 정보에 대한 중심어(keyword) 검색뿐만 아니라 전문(Full-Text)에 대한 검색 요구가 일반화되었다. 본 연구에서는 전문 검색을 위한 설계 방안을 제안한다. 기존에 제안된 전문 검색 방안과 오라클에서 제공하는 interMedia Text를 이용한 전문 검색 방안을 비교한다. 이를 기반으로 정보 검색 시스템에서 구현 방안을 제시한다.

  • PDF

PubMine: An Ontology-Based Text Mining System for Deducing Relationships among Biological Entities

  • Kim, Tae-Kyung;Oh, Jeong-Su;Ko, Gun-Hwan;Cho, Wan-Sup;Hou, Bo-Kyeng;Lee, Sang-Hyuk
    • Interdisciplinary Bio Central
    • /
    • 제3권2호
    • /
    • pp.7.1-7.6
    • /
    • 2011
  • Background: Published manuscripts are the main source of biological knowledge. Since the manual examination is almost impossible due to the huge volume of literature data (approximately 19 million abstracts in PubMed), intelligent text mining systems are of great utility for knowledge discovery. However, most of current text mining tools have limited applicability because of i) providing abstract-based search rather than sentence-based search, ii) improper use or lack of ontology terms, iii) the design to be used for specific subjects, or iv) slow response time that hampers web services and real time applications. Results: We introduce an advanced text mining system called PubMine that supports intelligent knowledge discovery based on diverse bio-ontologies. PubMine improves query accuracy and flexibility with advanced search capabilities of fuzzy search, wildcard search, proximity search, range search, and the Boolean combinations. Furthermore, PubMine allows users to extract multi-dimensional relationships between genes, diseases, and chemical compounds by using OLAP (On-Line Analytical Processing) techniques. The HUGO gene symbols and the MeSH ontology for diseases, chemical compounds, and anatomy have been included in the current version of PubMine, which is freely available at http://pubmine.kobic.re.kr. Conclusions: PubMine is a unique bio-text mining system that provides flexible searches and analysis of biological entity relationships. We believe that PubMine would serve as a key bioinformatics utility due to its rapid response to enable web services for community and to the flexibility to accommodate general ontology.

이미지 인지 유형 및 검색질의 방식에 따른 검색 효율성에 관한 연구 (A Study on the Retrieval Effectiveness Based on Image Query Types)

  • 김성희;이근영
    • 한국문헌정보학회지
    • /
    • 제47권3호
    • /
    • pp.321-342
    • /
    • 2013
  • 본 연구에서는 이미지 인지유형 및 질의방식에 따른 검색방법의 효율성을 분석하기 위해 32명의 대학생들이 구글 이미지 검색시스템을 이용하여 검색실험을 실시하였다. 이미지 인지유형은 구체적(specific), 일반적(generic), 추상적(abstract) 유형으로 구분하였으며, 각 유형별 이미지를 텍스트검색, 예제에 따른 검색(QBE: Query by example), 하이브리드검색 등 3가지 질의방식으로 구분하여 실험을 실시하였다. 독립변수는 이미지 인지유형 및 질의방식이며 종속변수는 검색된 적합한 이미지의 수이다. 데이터 분석은 일원배치 분산분석(One-way ANOVA)과 이원배치분석(Two way ANOVA)을 이용하여 검증하였다. 분석결과로는 구체적 이미지와 일반적 이미지 인지유형에서는 텍스트 및 하이브리드 방식이 검색효율성이 높게 나타났고 추상적 이미지 인지유형에서는 QBE이 검색효율성이 높은 것으로 나타났다. 본 연구 결과는 이미지 검색에서 검색효율성을 높이기 위한 방안을 마련하는데 기초자료로 활용될 수 있을 것이다.

문자열 검출을 위한 슬라브 영역 추정 (Slab Region Localization for Text Extraction using SIFT Features)

  • 최종현;최성후;윤종필;구근휘;김상우
    • 전기학회논문지
    • /
    • 제58권5호
    • /
    • pp.1025-1034
    • /
    • 2009
  • In steel making production line, steel slabs are given a unique identification number. This identification number, Slab management number(SMN), gives information about the use of the slab. Identification of SMN has been done by humans for several years, but this is expensive and not accurate and it has been a heavy burden on the workers. Consequently, to improve efficiency, automatic recognition system is desirable. Generally, a recognition system consists of text localization, text extraction, character segmentation, and character recognition. For exact SMN identification, all the stage of the recognition system must be successful. In particular, the text localization is great important stage and difficult to process. However, because of many text-like patterns in a complex background and high fuzziness between the slab and background, directly extracting text region is difficult to process. If the slab region including SMN can be detected precisely, text localization algorithm will be able to be developed on the more simple method and the processing time of the overall recognition system will be reduced. This paper describes about the slab region localization using SIFT(Scale Invariant Feature Transform) features in the image. First, SIFT algorithm is applied the captured background and slab image, then features of two images are matched by Nearest Neighbor(NN) algorithm. However, correct matching rate can be low when two images are matched. Thus, to remove incorrect match between the features of two images, geometric locations of the matched two feature points are used. Finally, search rectangle method is performed in correct matching features, and then the top boundary and side boundaries of the slab region are determined. For this processes, we can reduce search region for extraction of SMN from the slab image. Most cases, to extract text region, search region is heuristically fixed [1][2]. However, the proposed algorithm is more analytic than other algorithms, because the search region is not fixed and the slab region is searched in the whole image. Experimental results show that the proposed algorithm has a good performance.

실시간 속기 자막 환경에서 멀티미디어 정보 검색을 위한 Prefix Array (The Prefix Array for Multimedia Information Retrieval in the Real-Time Stenograph)

  • 김동주;김한우
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 2006년 학술대회 논문집 정보 및 제어부문
    • /
    • pp.521-523
    • /
    • 2006
  • This paper proposes an algorithm and its data structure to support real-time full-text search for the streamed or broadcasted multimedia data containing real-time stenograph text. Since the traditional indexing method used at information retrieval area uses the linguistic information, there is a heavy cost. Therefore, we propose the algorithm and its data structure based on suffix array, which is a simple data structure and has low space complexity. Suffix array is useful frequently to search for huge text. However, subtitle text of multimedia data is to get longer by time. Therefore, suffix array must be reconstructed because subtitle text is continually changed. We propose the data structure called prefix array and search algorithm using it.

  • PDF

An Optimized e-Lecture Video Search and Indexing framework

  • Medida, Lakshmi Haritha;Ramani, Kasarapu
    • International Journal of Computer Science & Network Security
    • /
    • 제21권8호
    • /
    • pp.87-96
    • /
    • 2021
  • The demand for e-learning through video lectures is rapidly increasing due to its diverse advantages over the traditional learning methods. This led to massive volumes of web-based lecture videos. Indexing and retrieval of a lecture video or a lecture video topic has thus proved to be an exceptionally challenging problem. Many techniques listed by literature were either visual or audio based, but not both. Since the effects of both the visual and audio components are equally important for the content-based indexing and retrieval, the current work is focused on both these components. A framework for automatic topic-based indexing and search depending on the innate content of the lecture videos is presented. The text from the slides is extracted using the proposed Merged Bounding Box (MBB) text detector. The audio component text extraction is done using Google Speech Recognition (GSR) technology. This hybrid approach generates the indexing keywords from the merged transcripts of both the video and audio component extractors. The search within the indexed documents is optimized based on the Naïve Bayes (NB) Classification and K-Means Clustering models. This optimized search retrieves results by searching only the relevant document cluster in the predefined categories and not the whole lecture video corpus. The work is carried out on the dataset generated by assigning categories to the lecture video transcripts gathered from e-learning portals. The performance of search is assessed based on the accuracy and time taken. Further the improved accuracy of the proposed indexing technique is compared with the accepted chain indexing technique.

딥러닝 모델을 활용한 실시간 인쇄물 문자 탐지 시스템 (Real-time Printed Text Detection System using Deep Learning Model)

  • 최예준;김송원;문미경
    • 한국전자통신학회논문지
    • /
    • 제19권3호
    • /
    • pp.523-530
    • /
    • 2024
  • 웹페이지나 디지털 문서 등과 같은 온라인에서는 사용자가 검색하고 싶은 특정 단어나 특정 문구를 실시간으로 검색하는 기능이 있다. 인쇄된 도서나 참고서 등과 같은 인쇄물에는 실시간으로 특정 단어나 특정 문구를 찾는 기능이 없어 어려움을 겪는 경우가 많다. 본 논문에서는 텍스트를 탐지(Detection)하는 딥러닝 모델과 텍스트를 인식(Recognition)하는 OCR을 활용한 실시간 문자 탐지 시스템의 개발내용에 관해 기술한다. 본 연구에서는 EAST 모델을 사용하여 텍스트를 탐지하는 방법, 탐지한 텍스트를 EasyOCR을 사용하여 인식하는 방법, 인식한 텍스트를 사용자가 검색하고 싶은 특정 단어나 특정 문구를 비교하여 bounding box로 나타내는 방법을 제안한다. 이 시스템을 통해 사용자는 도서나 참고서 등과 같은 인쇄물에서 실시간으로 검색하고 싶은 특정 단어나 특정 문구를 찾아 필요한 정보를 쉽고 빠르게 찾는 것에 효과적일 것을 기대한다.