• Journal of KIISE:Databases
• /
• v.36 no.6
• /
• pp.455-465
• /
• 2009

Suffix trees are widely used in similar sequence matching for DNA. They have several problems such as time consuming, large space usages of disks and memories and data skew, since DNA sequences are very large and do not fit in the main memory. Thus, in the paper, we present a space efficient indexing method called SENoM, allowing us to build trees without merging phases for the partitioned sub trees. The proposed method is constructed in two phases. In the first phase, we partition the suffixes of the input string based on a common variable-length prefix till the number of suffixes is smaller than a threshold. In the second phase, we construct a sub tree based on the disk using the suffix sets, and then write it to the disk. The proposed method, SENoM eliminates complex merging phases. We show experimentally that proposed method is effective as bellows. SENoM reduces the disk usage less than 35% and reduces the memory usage less than 20% compared with TRELLIS algorithm. SENoM is available to query efficiently using the prefix tree even when the length of query sequence is large.

• The KIPS Transactions:PartD
• /
• v.10D no.3
• /
• pp.501-512
• /
• 2003

This paper proposes an indexing technique for fast retrieval of similar image subsequences using the multi-dimensional time warping distance. The time warping distance is a more suitable similarity measure than Lp distance in many applications where sequences may be of different lengths and/or different sampling rates. Our indexing scheme employs a disk-based suffix tree as an index structure and uses a lower-bound distance function to filter out dissimilar subsequences without false dismissals. It applies the normaliration for an easier control of relative weighting of feature dimensions and the discretization to compress the index tree. Experiments on medical and synthetic image sequences verify that the proposed method significantly outperforms the naive method and scales well in a large volume of image sequence databases.

• Journal of KIISE:Databases
• /
• v.34 no.2
• /
• pp.119-132
• /
• 2007

In molecular biology, approximate subsequence search is one of the most important operations. In this paper, we propose an accurate and efficient method for approximate subsequence search in large DNA databases. The proposed method basically adopts a binary trie as its primary structure and stores all the window subsequences extracted from a DNA sequence. For approximate subsequence search, it traverses the binary trie in a breadth-first fashion and retrieves all the matched subsequences from the traversed path within the trie by a dynamic programming technique. However, the proposed method stores only window subsequences of the pre-determined length, and thus suffers from large post-processing time in case of long query sequences. To overcome this problem, we divide a query sequence into shorter pieces, perform searching for those subsequences, and then merge their results. To verify the superiority of the proposed method, we conducted performance evaluation via a series of experiments. The results reveal that the proposed method, which requires smaller storage space, achieves 4 to 17 times improvement in performance over the suffix tree based method. Even when the length of a query sequence is large, our method is more than an order of magnitude faster than the suffix tree based method and the Smith-Waterman algorithm.

• Proceedings of the Korean Information Science Society Conference
• /
• 2007.10a
• /
• pp.267-268
• /
• 2007

• Proceedings of the Korean Information Science Society Conference
• /
• 2004.10a
• /
• pp.712-714
• /
• 2004

본 논문에서는 wildcard character를 포함하는 문자열의 집합을 대상으로, 이들 사이의 subsumption 관계를 파악하여 더 구체적인 정보를 가지는 문자열들의 집합을 구하고자 하는 것이다. 이를 위해 기존의 suffix tree 알고리즘이 wildcard character를 포함하는 문자열을 처리할 수 있도록 단순 적용한 방법과 trie의 집합을 이용하여 wildcard character를 포함한 문자열을 처리하는 두 가지 방법을 고려하였다

• Proceedings of the Korea Information Processing Society Conference
• /
• 2009.04a
• /
• pp.996-998
• /
• 2009

접미사 트리는 주어진 하나의 문자열의 모든 접미사를 표현하는 트리로, 문자열 처리, 압축 등 다양한 분야에서 활용된다. 접미사 트리는 문자열 집합에 대한 자료구조로 확장될 수 있는데, 이를 일반화된 접미사 트리라 부른다. 본 논문에서는 일반화된 접미사 트리를 동반적이면서 온라인으로 생성하는 문제를 다룬다. 기존의 생성 알고리즘은 정방향의 문자열이 아닌 역방향의 문자열들에 대한 일반화된 접미사 트리를 생성하여, 부자연스럽다. 본 논문에서는 정방향 문자열들의 일반화된 접미사 트리를 동반적이면서 온라인으로 생성하는 알고리즘을 제시한다.

• Journal of Creative Information Culture
• /
• v.7 no.3
• /
• pp.157-165
• /
• 2021

As the use of various application services utilizing Web and IoT and the need for large amounts of data management expand accordingly, the importance of efficient data expression and exchange scheme and data query processing is increasing. JSON, characterized by its simplicity, is being used in various fields as a format for data exchange and data storage instead of XML, which is a standard data expression and exchange language on the Web. This means that it is important to develop indexing and query processing techniques to effectively access and search large amounts of data expressed in JSON. Therefore, in this paper, we modeled JSON data with a hierarchical structure in a tree form, and proposed indexing and query processing using the path concept. In particular, we designed an index structure using a suffix array widely used in text search and introduced simple and complex path-based JSON data query processing methods.

• Journal of KIISE:Databases
• /
• v.31 no.6
• /
• pp.650-663
• /
• 2004

In molecular biology, DNA sequence searching is one of the most crucial operations. Since DNA databases contain a huge volume of sequences, a fast indexing mechanism is essential for efficient processing of DNA sequence searches. In this paper, we first identify the problems of the suffix tree in aspects of the storage overhead, search performance, and integration with DBMSs. Then, we propose a new index structure that solves those problems. The proposed index consists of two parts: the primary part represents the trie as bit strings without any pointers, and the secondary part helps fast accesses of the leaf nodes of the trio that need to be accessed for post processing. We also suggest an efficient algorithm based on that index for DNA sequence searching. To verify the superiority of the proposed approach, we conducted a performance evaluation via a series of experiments. The results revealed that the proposed approach, which requires smaller storage space, achieves 13 to 29 times performance improvement over the suffix tree.

• Journal of Korea Artificial Intelligence Association
• /
• v.1 no.2
• /
• pp.1-6
• /
• 2023

In this paper, we present a new method for classifying malicious URLs to reduce cases of learning difficulties due to unfamiliar and difficult terms related to information protection. This study plans to extract only visually distinguishable features within the URL structure and compare them through map learning algorithms, and to compare the contribution values of the best map learning algorithm methods to extract features that have the most impact on classifying malicious URLs. As research data, Kaggle used data that classified 7,046 malicious URLs and 7.046 normal URLs. As a result of the study, among the three supervised learning algorithms used (Decision Tree, Support Vector Machine, and Logistic Regression), the Decision Tree algorithm showed the best performance with 83% accuracy, 83.1% F1-score and 83.6% Recall values. It was confirmed that the contribution value of https is the highest among whether to use https, sub domain, and prefix and suffix, which can be visually distinguished through the feature contribution of Decision Tree. Although it has been difficult to learn unfamiliar and difficult terms so far, this study will be able to provide an intuitive judgment method without explanation of the terms and prove its usefulness in the field of malicious URL detection.

• Journal of KIISE:Computer Systems and Theory
• /
• v.31 no.3_4
• /
• pp.145-151
• /
• 2004

We present an efficient data structure to obtain the range minima in an away in constant time. Recently, suffix ways are extensively used to search DNA sequences fast in bioinformatics. In constructing suffix arrays, solving the range minima problem is necessary When we construct suffix arrays, we should solve the range minima problem not only in a time-efficient way but also in a space-efficient way. The reason is that DNA sequences consist of millions or billions of bases. Until now, the most efficient data structure to find the range minima in an way in constant time is based on the method that converts the range minima problem in an array into the LCA (Lowest Common Ancestor) problem in a Cartesian tree and then converts the LCA problem into the range minima problem in a specific array. This data structure occupies O( n) space and is constructed in O(n) time. However since this data structure includes intermediate data structures required to convert the range minima problem in an array into other problems, it requires large space (=13n) and much time. Our data structure is based on the method that directly solves the range minima problem. Thus, our data structure requires small space (=5n) and less time in practice. As a matter of course, our data structure requires O(n) time and space theoretically.