• Title/Summary/Keyword: Indexing Technique

Search Result 203, Processing Time 0.034 seconds

Design and Evaluation of a High-performance Key-value Storage for Industrial IoT Environments (산업용 IoT 환경을 위한 고성능 키-값 저장소의 설계 및 평가)

  • Han, Hyuck
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.7
    • /
    • pp.127-133
    • /
    • 2021
  • In industrial IoT environments, sensors generate data for their detection targets and deliver the data to IoT gateways. Therefore, managing large amounts of real-time sensor data is an essential feature for IoT gateways, and key-value storage engines are widely used to manage these sensor data. However, key-value storage engines used in IoT gateways do not take into account the characteristics of sensor data generated in industrial IoT environments, and this limits the performance of key-value storage engines. In this paper, we optimize the key-value storage engine by utilizing the features of sensor data in industrial IoT environments. The proposed optimization technique is to analyze the key, which is the input of a key-value storage engine, for further indexing. This reduces excessive write amplification and improves performance. We implement our optimization scheme in LevelDB and use the workload of the TPCx-IoT benchmark to evaluate our proposed scheme. From experimental results we show that our proposed technique achieves up to 21 times better than the existing scheme, and this shows that the proposed technique can perform high-speed data ingestion in industrial IoT environments.

Font Classification of English Printed Character using Non-negative Matrix Factorization (NMF를 이용한 영문자 활자체 폰트 분류)

  • Lee, Chang-Woo;Kang, Hyun;Jung, Kee-Chul;Kim, Hang-Joon
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.41 no.2
    • /
    • pp.65-76
    • /
    • 2004
  • Today, most documents are electronically produced and their paleography is digitalized by imaging, resulting in a tremendous number of electronic documents in the shape of images. Therefore, to process these document images, many methods of document structure analysis and recognition have already been proposed, including font classification. Accordingly, the current paper proposes a font classification method for document images that uses non-negative matrix factorization (NMF), which is able to learn part-based representations of objects. In the proposed method, spatially total features of font images are automatically extracted using NMF, then the appropriateness of the features specifying each font is investigated. The proposed method is expected to improve the performance of optical character recognition (OCR), document indexing, and retrieval systems, when such systems adopt a font classifier as a preprocessor.

Semantic Event Detection and Summary for TV Golf Program Using MPEG-7 Descriptors (MPEG-7 기술자를 이용한 TV 골프 프로그램의 이벤트검출 및 요약)

  • 김천석;이희경;남제호;강경옥;노용만
    • Journal of Broadcast Engineering
    • /
    • v.7 no.2
    • /
    • pp.96-106
    • /
    • 2002
  • We introduce a novel scheme to characterize and index events in TV golf programs using MPEG-7 descriptors. Our goal is to identify and localize the golf events of interest to facilitate highlight-based video indexing and summarization. In particular, we analyze multiple (low-level) visual features using domain-specific model to create a perceptual relation for semantically meaningful(high-level) event identification. Furthermore, we summarize a TV golf program with TV-Anytime segmentation metadata, a standard form of an XML-based metadata description, in which the golf events are represented by temporally localized segments and segment groups of highlights. Experimental results show that our proposed technique provides reasonable performance for identifying a variety of golf events.

High-Speed Search Mechanism based on B-Tree Index Vector for Huge Web Log Mining and Web Attack Detection (대용량 웹 로그 마이닝 및 공격탐지를 위한 B-트리 인덱스 벡터 기반 고속 검색 기법)

  • Lee, Hyung-Woo;Kim, Tae-Su
    • Journal of Korea Multimedia Society
    • /
    • v.11 no.11
    • /
    • pp.1601-1614
    • /
    • 2008
  • The number of web service users has been increased rapidly as existing services are changed into the web-based internet applications. Therefore, it is necessary for us to use web log pre-processing technique to detect attacks on diverse web service transactions and it is also possible to extract web mining information. However, existing mechanisms did not provide efficient pre-processing procedures for a huge volume of web log data. In this paper, we proposed both a field based parsing and a high-speed log indexing mechanism based on the suggested B-tree Index Vector structure for performance enhancement. In experiments, the proposed mechanism provides an efficient web log pre-processing and search functions with a session classification. Therefore it is useful to enhance web attack detection function.

  • PDF

Massive Terrain Rendering Method Using RGBA Channel Indexing of Wavelet Coefficients (웨이블릿 압축 계수의 RGBA채널 인덱싱을 이용한 대용량 지형 렌더링 기법)

  • Kim, Tae-Gwon;Lee, Eun-Seok;Shin, Byeong-Seok
    • Journal of Korea Game Society
    • /
    • v.13 no.5
    • /
    • pp.55-62
    • /
    • 2013
  • Since large terrain data can not be loaded on the GPU or CPU memory at once, out-of-core methods which read necessary part from the secondary storage such as a hard disk are commonly used. However, long delay may occur due to limited bandwidth while loading the data from the hard disk to memory. We propose efficient rendering method of large terrain data, which compresses the data with wavelet technique and save its coefficients in RGBA channel of an image us, then decompresses that in rendering stage. Entire process is performed in GPU using Direct Compute. By reducing the amount of data transfer, performing wavelet computations in parallel and doing decompression quickly on the GPU, our method can reduce rendering time effectively.

A Design and Implementation of Dynamic Hybrid P2P System with Hierarchical Group Management and Maintenance of Reliability (계층적 그룹관리와 신뢰성을 위한 동적인 변형 P2P 시스템 설계 및 구현)

  • Lee, Seok-Hee;Cho, Sang;Kim, Sung-Yeol
    • The KIPS Transactions:PartD
    • /
    • v.11D no.4
    • /
    • pp.975-982
    • /
    • 2004
  • In current P2P concept, pure P2P and Hybrid P2P structures are used commonly. Gnutella and Ktella are forms of pure P2P. and forms of Hybrid P2P are innumerable. File searching models exist in these models. These models provide group management for file sharing, searching and indexing. The general file sharing model is good at maintaining connectivity. However, it is defective in group management. Therefore, this study approaches hierarchical structure in file sharing models through routing technique and backup system. This system was designed so that the user was able to maintain group efficiency and connection reliability in large-scale network.

Automatic Extraction of Focused Video Object from Low Depth-of-Field Image Sequences (낮은 피사계 심도의 동영상에서 포커스 된 비디오 객체의 자동 검출)

  • Park, Jung-Woo;Kim, Chang-Ick
    • Journal of KIISE:Software and Applications
    • /
    • v.33 no.10
    • /
    • pp.851-861
    • /
    • 2006
  • The paper proposes a novel unsupervised video object segmentation algorithm for image sequences with low depth-of-field (DOF), which is a popular photographic technique enabling to represent the intention of photographer by giving a clear focus only on an object-of-interest (OOI). The proposed algorithm largely consists of two modules. The first module automatically extracts OOIs from the first frame by separating sharply focused OOIs from other out-of-focused foreground or background objects. The second module tracks OOIs for the rest of the video sequence, aimed at running the system in real-time, or at least, semi-real-time. The experimental results indicate that the proposed algorithm provides an effective tool, which can be a basis of applications, such as video analysis for virtual reality, immersive video system, photo-realistic video scene generation and video indexing systems.

Multi-aperture Photometry Pipeline for DEEP-South Data

  • Chang, Seo-Won;Byun, Yong-Ik;Kim, Myung-Jin;Moon, Hong-Kyu;Yim, Hong-Suh;Shin, Min-Su;Kang, Young-Woon
    • The Bulletin of The Korean Astronomical Society
    • /
    • v.41 no.1
    • /
    • pp.56.2-56.2
    • /
    • 2016
  • We present a multi-aperture photometry pipeline for DEEP-South (Deep Ecliptic Patrol of the Southern Sky) time-series data, written in C. The pipeline is designed to do robust high-precision photometry and calibration of non-crowded fields with a varying point-spread function, allowing for the wholesale search and characterization of both temporal and spatial variabilities. Our time-series photometry method consists of three parts: (i) extracting all point sources with several pixel/blind parameters, (ii) determining the optimized aperture for each source where we consider whether the measured flux within the aperture is contaminated by unwanted artifacts, and (iii) correcting position-dependent variations in the PSF shape across the mosaic CCD. In order to provide faster access to the resultant catalogs, we also utilize an efficient indexing technique using compressed bitmap indices (FastBit). Lastly, we focus on the development and application of catalog-based searches that aid the identification of high-probable single events from the indexed database. This catalog-based approach is still useful to identify new point-sources or moving objects in non-crowded fields. The performance of the pipeline is being tested on various sets of time-series data available in several archives: DEEP-South asteroid survey and HAT-South/MMT exoplanet survey data sets.

  • PDF

RDBMS Based Efficient Method for Shortest Path Searching Over Large Graphs Using K-degree Index Table (대용량 그래프에서 k-차수 인덱스 테이블을 이용한 RDBMS 기반의 효율적인 최단 경로 탐색 기법)

  • Hong, Jihye;Han, Yongkoo;Lee, Young-Koo
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.5
    • /
    • pp.179-186
    • /
    • 2014
  • Current networks such as social network, web page link, traffic network are big data which have the large numbers of nodes and edges. Many applications such as social network services and navigation systems use these networks. Since big networks are not fit into the memory, existing in-memory based analysis techniques cannot provide high performance. Frontier-Expansion-Merge (FEM) framework for graph search operations using three corresponding operators in the relational database (RDB) context. FEM exploits an index table that stores pre-computed partial paths for efficient shortest path discovery. However, the index table of FEM has low hit ratio because the indices are determined by distances of indices rather than the possibility of containing a shortest path. In this paper, we propose an method that construct index table using high degree nodes having high hit ratio for efficient shortest path discovery. We experimentally verify that our index technique can support shortest path discovery efficiently in real-world datasets.

The Information Retrieval System for Software Reuse (소프트웨어 재사용을 위한 정보검색시스템 구축)

  • Kim, Young-Geil
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.9 no.1
    • /
    • pp.1-8
    • /
    • 2016
  • In this paper, several problems functioning as the obstacles against software reuse were summarized. Among them, the issues dealt with in this paper include the effective method for constructing the library, the proper structure of the library, and the efficient retrieval technique. The knowledge-based approach and the information retrieval approach were integrated to construct and manage the library. The former is on the object- oriented model. Basically the object-oriented library is based on the classes and organized by inheritance. Because inheritance hierarchy is based on syntactical information, it dose not present the relationship of functionality. Using the information retrieval approach, the index file which characterizes the component and similarity among the components can be analyzed. Especially, we focused on the reusable library for the object-oriented programming environments.