• Title/Summary/Keyword: Suffix Tree

Search Result 50, Processing Time 0.034 seconds

Implementation and analysis of a parallel suffix tree construction algorithm using TBB and Cilk Plus (TBB, Cilk Plus를 이용한 병렬 접미사 트리 생성 알고리즘 구현 및 성능 분석)

  • Seo, Jun-Ho;Na, Joong-Chae
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2012.06a
    • /
    • pp.403-405
    • /
    • 2012
  • 접미사 트리는 문자열 압축, 텍스트 처리, 생물정보학 등 다양한 응용 분야에서 사용되는 인덱스 자료구조이다. 최근 64bit 하드웨어와 멀티코어 CPU가 보급됨에 따라 메모리상에서 병렬로 접미사 트리를 생성하는 알고리즘이 활발히 연구되고 있다. 본 논문에서는 McCreight의 선형시간 알고리즘과 Chen의 병렬 알고리즘을 기반으로 메모리상에서 접미사 트리를 병렬로 생성하는 구현 방법을 보였으며, TBB, Cilk Plus와 같은 병렬 프로그래밍 라이브러리를 이용하여 병렬 알고리즘을 구현하였다. 알고리즘 실험 결과 병렬로 수행한 알고리즘이 직렬로 수행한 결과보다 최대 4배 가량 성능 향상을 얻을 수 있었으며, 병렬 라이브러리를 사용함으로써 가지는 오버헤드는 극히 적은 것으로 나타났다.

Discovering and Matching Elastic Rules in Sequence Databases (시퀀스 데이터베이스에서 유연 규칙의 탐사 및 매칭)

  • ;Wesley Chu
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.26 no.7A
    • /
    • pp.1162-1169
    • /
    • 2001
  • 유연 패턴은 시간 축으로 확장 및 수축할 수 있는 요소들의 순서화된 리스트이다. 유연 패턴은 서로 다른 샘플링 비율을 갖는 데이터 시퀀스들로부터 규칙들을 찾아내는데 유용하게 사용된다. 본 연구에서는 헤드(head: 규칙의 왼쪽 부분)와 바디(body: 규칙의 오른쪽 부분)가 모두 유연 패턴으로 구성된 규칙들을 신속하게 찾도록 하기 위하여 데이터 시퀀스로부터 서픽스 트리(suffix tree)를 구성한다. 이 서픽스 트리는 유연 규칙들의 압축된 표현이며, 타깃 헤드 시퀀스와 매치되는 규칙을 찾기 위한 인덱스 구조로서 사용된다. 만일, 매치되는 규칙을 찾을 수 없는 경우에는 규칙 완화(rule relaxation)의 개념을 이용한다. 클러스터 계층(cluster hierarchy)과 완화 오차(relaxation error)를 사용하여 타깃 헤드 시퀀스의 고유한 정보를 대부분 포함하고 있는 최소한으로 완화된 규칙을 찾는다. 다양한 실험을 통한 성능 평가를 통하여 제안한 기법의 우수성을 검증한다.

  • PDF

Matching of Elastic Rules in Sequence Databases (시퀀스 데이터베이스를 위한 유연 규칙 매칭)

  • Park, Sang-Hyun;Chu, Wesley W.;Kim, Sang-Wook
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2001.04a
    • /
    • pp.57-60
    • /
    • 2001
  • 본 논문에서는 유연 패턴(elastic pattern)을 갖는 규칙(rule)을 탐사하고 매칭하는 기법에 대해 논의한다. 유연 패턴은 시간 축으로 확장 및 수축할 수 있는 요소들의 순서화된 리스트이다. 유연 패턴은 서로 다른 샘플링 비율을 갖는 데이터 시퀀스들로부터 규칙들을 찾아내는데 유용하게 사용된다. 본 연구에서는 헤드(head: 규칙의 왼쪽 부분)와 바디(body: 규칙의 오른쪽 부분)가 모두 유연 패턴으로 구성된 규칙들을 신속하게 찾도록 하기 위하여 데이터 시퀀스로부터 서픽스 트리(suffix tree)를 구성한다. 이 서픽스 트리는 유연 규칙들의 압축된 표현이며, 타깃 헤드 시퀀스와 매치되는 규칙을 찾기 위한 인덱스 구조로서 사용된다. 만일, 매치되는 규칙을 찾을 수 없는 경우에는 규칙 완화(rule relaxation)의 개념을 이용한다. 클러스터 계층(cluster hierarchy)과 완화 오차(relaxation error)를 사용하여 타깃 헤드 시퀀스의 고유한 정보를 대부분 포함하고 있는 최소한으로 완화된 규칙을 찾는다. 다양한 실험을 통한 성능 평가를 통하여 제안한 기법의 우수성을 검증한다.

  • PDF

A Phoneme-based Approximate String Searching System for Restricted Korean Character Input Environments (제한된 한글 입력환경을 위한 음소기반 근사 문자열 검색 시스템)

  • Yoon, Tai-Jin;Cho, Hwan-Gue;Chung, Woo-Keun
    • Journal of KIISE:Software and Applications
    • /
    • v.37 no.10
    • /
    • pp.788-801
    • /
    • 2010
  • Advancing of mobile device is remarkable, so the research on mobile input device is getting more important issue. There are lots of input devices such as keypad, QWERTY keypad, touch and speech recognizer, but they are not as convenient as typical keyboard-based desktop input devices so input strings usually contain many typing errors. These input errors are not trouble with communication among person, but it has very critical problem with searching in database, such as dictionary and address book, we can not obtain correct results. Especially, Hangeul has more than 10,000 different characters because one Hangeul character is made by combination of consonants and vowels, frequency of error is higher than English. Generally, suffix tree is the most widely used data structure to deal with errors of query, but it is not enough for variety errors. In this paper, we propose fast approximate Korean word searching system, which allows variety typing errors. This system includes several algorithms for applying general approximate string searching to Hangeul. And we present profanity filters by using proposed system. This system filters over than 90% of coined profanities.

RSP-DS: Real Time Sequential Patterns Analysis in Data Streams (RSP-DS: 데이터 스트림에서의 실시간 순차 패턴 분석)

  • Shin Jae-Jyn;Kim Ho-Seok;Kim Kyoung-Bae;Bae Hae-Young
    • Journal of Korea Multimedia Society
    • /
    • v.9 no.9
    • /
    • pp.1118-1130
    • /
    • 2006
  • Existed pattern analysis algorithms in data streams environment have researched performance improvement and effective memory usage. But when new data streams come, existed pattern analysis algorithms have to analyze patterns again and have to generate pattern tree again. This approach needs many calculations in real situation that needs real time pattern analysis. This paper proposes a method that continuously analyzes patterns of incoming data streams in real time. This method analyzes patterns fast, and thereafter obtains real time patterns by updating previously analyzed patterns. The incoming data streams are divided into several sequences based on time based window. Informations of the sequences are inputted into a hash table. When the number of the sequences are over predefined bound, patterns are analyzed from the hash table. The patterns form a pattern tree, and later created new patterns update the pattern tree. In this way, real time patterns are always maintained in the pattern tree. During pattern analysis, suffixes of both new pattern and existed pattern in the tree can be same. Then a pointer is created from the new pattern to the existed pattern. This method reduce calculation time during duplicated pattern analysis. And old patterns in the tree are deleted easily by FIFO method. The advantage of our algorithm is proved by performance comparison with existed method, MILE, in a condition that pattern is changed continuously. And we look around performance variation by changing several variable in the algorithm.

  • PDF

An Efficient Approach to Mining Maximal Contiguous Frequent Patterns from Large DNA Sequence Databases

  • Karim, Md. Rezaul;Rashid, Md. Mamunur;Jeong, Byeong-Soo;Choi, Ho-Jin
    • Genomics & Informatics
    • /
    • v.10 no.1
    • /
    • pp.51-57
    • /
    • 2012
  • Mining interesting patterns from DNA sequences is one of the most challenging tasks in bioinformatics and computational biology. Maximal contiguous frequent patterns are preferable for expressing the function and structure of DNA sequences and hence can capture the common data characteristics among related sequences. Biologists are interested in finding frequent orderly arrangements of motifs that are responsible for similar expression of a group of genes. In order to reduce mining time and complexity, however, most existing sequence mining algorithms either focus on finding short DNA sequences or require explicit specification of sequence lengths in advance. The challenge is to find longer sequences without specifying sequence lengths in advance. In this paper, we propose an efficient approach to mining maximal contiguous frequent patterns from large DNA sequence datasets. The experimental results show that our proposed approach is memory-efficient and mines maximal contiguous frequent patterns within a reasonable time.

Shape-Based Subsequence Retrieval Supporting Multiple Models in Time-Series Databases (시계열 데이터베이스에서 복수의 모델을 지원하는 모양 기반 서브시퀀스 검색)

  • Won, Jung-Im;Yoon, Jee-Hee;Kim, Sang-Wook;Park, Sang-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.10D no.4
    • /
    • pp.577-590
    • /
    • 2003
  • The shape-based retrieval is defined as the operation that searches for the (sub) sequences whose shapes are similar to that of a query sequence regardless of their actual element values. In this paper, we propose a similarity model suitable for shape-based retrieval and present an indexing method for supporting the similarity model. The proposed similarity model enables to retrieve similar shapes accurately by providing the combination of various shape-preserving transformations such as normalization, moving average, and time warping. Our indexing method stores every distinct subsequence concisely into the disk-based suffix tree for efficient and adaptive query processing. We allow the user to dynamically choose a similarity model suitable for a given application. More specifically, we allow the user to determine the parameter p of the distance function $L_p$ when submitting a query. The result of extensive experiments revealed that our approach not only successfully finds the subsequences whose shapes are similar to a query shape but also significantly outperforms the sequence search.

Analysis of Accounts Receivable Aging Using Variable Order Markov Model (가변 마코프 모델을 활용한 매출 채권 연령 분석)

  • Kang, Yuncheol;Kang, Minji;Chung, Kwanghun
    • The Journal of Society for e-Business Studies
    • /
    • v.24 no.1
    • /
    • pp.91-103
    • /
    • 2019
  • An accurate prediction on near-future cash flows plays an important role for a company to attenuate the shortage risk of cash flow by preparing a plan for future investment in advance. Unfortunately, there exists a high level of uncertainty in the types of transactions that occur in the form of receivables in inter-company transactions, unlike other types of transactions, thereby making the prediction of cash flows difficult. In this study, we analyze the trend of cash flow related to account receivables that may arise between firms, by using a stochastic approach. In particular, we utilize Variable Order Markov (VOM) model to predict how future cash flows will change based on cash flow history. As a result of this study, we show that the average accuracy of the VOM model increases about 12.5% or more compared with that of other existing techniques.

A DNA Index Structure using Frequency and Position Information of Genetic Alphabet (염기문자의 빈도와 위치정보를 이용한 DNA 인덱스구조)

  • Kim Woo-Cheol;Park Sang-Hyun;Won Jung-Im;Kim Sang-Wook;Yoon Jee-Hee
    • Journal of KIISE:Databases
    • /
    • v.32 no.3
    • /
    • pp.263-275
    • /
    • 2005
  • In a large DNA database, indexing techniques are widely used for rapid approximate sequence searching. However, most indexing techniques require a space larger than original databases, and also suffer from difficulties in seamless integration with DBMS. In this paper, we suggest a space-efficient and disk-based indexing and query processing algorithm for approximate DNA sequence searching, specially exact match queries, wildcard match queries, and k-mismatch queries. Our indexing method places a sliding window at every possible location of a DNA sequence and extracts its signature by considering the occurrence frequency of each nucleotide. It then stores a set of signatures using a multi-dimensional index, such as R*-tree. Especially, by assigning a weight to each position of a window, it prevents signatures from being concentrated around a few spots in index space. Our query processing algorithm converts a query sequence into a multi-dimensional rectangle and searches the index for the signatures overlapped with the rectangle. The experiments with real biological data sets revealed that the proposed method is at least three times, twice, and several orders of magnitude faster than the suffix-tree-based method in exact match, wildcard match, and k- mismatch, respectively.

An Effective Similarity Search Technique supporting Time Warping in Sequence Databases (시퀀스 데이타베이스에서 타임 워핑을 지원하는 효과적인 유살 검색 기법)

  • Kim, Sang-Wook;Park, Sang-Hyun
    • Journal of KIISE:Databases
    • /
    • v.28 no.4
    • /
    • pp.643-654
    • /
    • 2001
  • This paper discusses an effective processing of similarity search that supports time warping in large sequence database. Time warping enables finding sequences with similar patterns even when they are of different length, Previous methods fail to employ multi-dimensional indexes without false dismissal since the time warping distance does not satisfy the triangular inequality. They have to scan all the database, thus suffer from serious performance degradation in large database. Another method that hires the suffix tree also shows poor performance due to the large tree size. In this paper we propose a new novel method for similarity search that supports time warping Our primary goal is to innovate on search performance in large database without false dismissal. to attain this goal ,we devise a new distance function $D_{tw-Ib}$ consistently underestimates the time warping distance and also satisfies the triangular inequality, $D_{tw-Ib}$ uses a 4-tuple feature vector extracted from each sequence and is invariant to time warping, For efficient processing, we employ a distance function, We prove that our method does not incur false dismissal. To verify the superiority of our method, we perform extensive experiments . The results reveal that our method achieves significant speedup up to 43 times with real-world S&P 500 stock data and up to 720 times with very large synthetic data.

  • PDF