• Title/Summary/Keyword: string processing

Search Result 140, Processing Time 0.025 seconds

Word Similarity Calculation by Using the Edit Distance Metrics with Consonant Normalization

  • Kang, Seung-Shik
    • Journal of Information Processing Systems
    • /
    • v.11 no.4
    • /
    • pp.573-582
    • /
    • 2015
  • Edit distance metrics are widely used for many applications such as string comparison and spelling error corrections. Hamming distance is a metric for two equal length strings and Damerau-Levenshtein distance is a well-known metrics for making spelling corrections through string-to-string comparison. Previous distance metrics seems to be appropriate for alphabetic languages like English and European languages. However, the conventional edit distance criterion is not the best method for agglutinative languages like Korean. The reason is that two or more letter units make a Korean character, which is called as a syllable. This mechanism of syllable-based word construction in the Korean language causes an edit distance calculation to be inefficient. As such, we have explored a new edit distance method by using consonant normalization and the normalization factor.

Comparing String Similarity Algorithms for Recognizing Task Names Found in Construction Documents (문자열 유사도 알고리즘을 이용한 공종명 인식의 자연어처리 연구 - 공종명 문자열 유사도 알고리즘의 비교 -)

  • Jeong, Sangwon;Jeong, Kichang
    • Korean Journal of Construction Engineering and Management
    • /
    • v.21 no.6
    • /
    • pp.125-134
    • /
    • 2020
  • Natural language encountered in construction documents largely deviates from those that are recommended by the authorities. Such practice that is lacking in coherence will discourage integrated research with automation, and it will hurt the productivity in the industry for the long run. This research aims to compare multiple string similarity (string matching) algorithms to compare each algorithm's performance in recognizing the same task name written in multiple different ways. We also aim to start a debate on how prevalent the aforementioned deviation is. Finally, we composed a small dataset that associates construction task names found in practice with the corresponding task names that are less cluttered w.r.t their formatting. We expect that this dataset can be used to validate future natural language processing approaches.

The Recognition of Occluded 2-D Objects Using the String Matching and Hash Retrieval Algorithm (스트링 매칭과 해시 검색을 이용한 겹쳐진 이차원 물체의 인식)

  • Kim, Kwan-Dong;Lee, Ji-Yong;Lee, Byeong-Gon;Ahn, Jae-Hyeong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.5 no.7
    • /
    • pp.1923-1932
    • /
    • 1998
  • This paper deals with a 2-D objects recognition algorithm. And in this paper, we present an algorithm which can reduce the computation time in model retrieval by means of hashing technique instead of using the binary~tree method. In this paper, we treat an object boundary as a string of structural units and use an attributed string matching algorithm to compute similarity measure between two strings. We select from the privileged strings a privileged string wIth mmimal eccentricity. This privileged string is treated as the reference string. And thell we wllstructed hash table using the distance between privileged string and the reference string as a key value. Once the database of all model strings is built, the recognition proceeds by segmenting the scene into a polygonal approximation. The distance between privileged string extracted from the scene and the reference string is used for model hypothesis rerieval from the table. As a result of the computer simulation, the proposed method can recognize objects only computing, the distance 2-3tiems, while previous method should compute the distance 8-10 times for model retrieval.

  • PDF

Connected Korean Digit Speech Recognition Using Vowel String and Number of Syllables (음절수와 모음 열을 이용한 한국어 연결 숫자 음성인식)

  • Youn, Jeh-Seon;Hong, Kwang-Seok
    • The KIPS Transactions:PartA
    • /
    • v.10A no.1
    • /
    • pp.1-6
    • /
    • 2003
  • In this paper, we present a new Korean connected digit recognition based on vowel string and number of syllables. There are two steps to reduce digit candidates. The first one is to determine the number and interval of digit. Once the number and interval of digit are determined, the second is to recognize the vowel string in the digit string. The digit candidates according to vowel string are recognized based on CV (consonant vowel), VCCV and VC unit HMM. The proposed method can cope effectively with the coarticulation effects and recognize the connected digit speech very well.

Design and Performance Evaluation of an Indexing Method for Partial String Searches (문자열 부분검색을 위한 색인기법의 설계 및 성능평가)

  • Gang, Seung-Heon;Yu, Jae-Su
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.6
    • /
    • pp.1458-1467
    • /
    • 1999
  • Existing index structures such as extendable hashing and B+-tree do not support partial string searches perfectly. The inverted file method and the signature file method that are used in the web retrieval engine also have problems that they do not provide partial string searches and suffer from serious retrieval performance degradation respectively. In this paper, we propose an efficient index method that supports partial string searches and achieves good retrieval performance. The proposed index method is based on the Inverted file structure. It constructs the index file with patterns that result from dividing terms by two syllables to support partial string searches. We analyze the characteristics of our proposed method through simulation experiments using wide range of parameter values. We analyze the derive analytic performance evaluation models of the existing inverted file method, signature file method and the proposed index method in terms of retrieval time and storage overhead. We show through performance comparison based on analytic models that the proposed method significantly improves retrieval performance over the existing method.

  • PDF

String extraction from text-background mixed documents using mathematical morphology (텍스트-배경무늬 혼합문서로부터 수리형태학을 이용한 문자열 추출)

  • 성연진;어진우
    • Journal of the Korean Institute of Telematics and Electronics S
    • /
    • v.34S no.10
    • /
    • pp.104-111
    • /
    • 1997
  • It is known as a difficult problem to recognize text-background mixed documents. In this paper a new string extraction algorithm, using mathematical morphology for the document consisting of text and overlapped periodic background pattern, is proposed. The algorithm consists of pattern periodicity feature extraction and background removal. The extracted pattern periodicity feature is used to determine the shape of structuring elements for morphological pre- and post-processing to remove background. The effectiveness of the proposed algorithm over the existing one is also verified through the experiments with various test documents.

  • PDF

Sound Synthesis of Gayageum by Impulse Responses of Body and Anjok (안족과 몸통의 임펄스 응답을 이용한 가야금 사운드 합성)

  • Cho Sang-Jin;Choi Gin-Kyu;Chong Ui-Pil
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.7 no.3
    • /
    • pp.102-107
    • /
    • 2006
  • In this paper, we propose a method of a sound synthesis of Korean plucked string instrument, gayageum, by physical modeling which use impulse responses of body and Anjok. Gayageum consists of three kinds of systems: string, body, and Anjok. These are a serial combination of linear time invariant systems. String can be modeled by digital delay line. Body and Anjok can be estimated by their impulse responses. We found three resonance frequencies in the body impulse response, and implemented resonator as body. Anjok was implemented as high pass filter in fundamental frequency band of gayageum. RMSEs of synthesized sounds are distributed from 0.01 to 0.03. It was difficult to distinguish the resulting synthesized sounds from the originals sound by ear.

  • PDF

Retargetable Intermediate Code Optimization System Using Tree Pattern Matching Techniques (트리패턴매칭기법의 재목적 가능한 중간코드 최적화 시스템)

  • Kim, Jeong-Suk;O, Se-Man
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.8
    • /
    • pp.2253-2261
    • /
    • 1999
  • ACK generates optimized code using the string pattern matching technique in pattern table generator and peephole optimizer. But string pattern matching method is not effective due to the many comparative actions in pattern selection. We designed and implemented the EM intermediate code optimizer using tree pattern matching algorithm composed of EM tree generator, optimization pattern table generator and tree pattern matcher. Tree pattern matching algorithm practices the pattern matching that centering around root node with refer to the pattern table, with traversing the EM tree by top-down method. As a result, compare to ACK string pattern matching methods, we found that the optimized code effected to pattern selection time, and contributed to improved the pattern selection time by about 10.8%.

  • PDF

Developing JSequitur to Study the Hierarchical Structure of Biological Sequences in a Grammatical Inference Framework of String Compression Algorithms

  • Galbadrakh, Bulgan;Lee, Kyung-Eun;Park, Hyun-Seok
    • Genomics & Informatics
    • /
    • v.10 no.4
    • /
    • pp.266-270
    • /
    • 2012
  • Grammatical inference methods are expected to find grammatical structures hidden in biological sequences. One hopes that studies of grammar serve as an appropriate tool for theory formation. Thus, we have developed JSequitur for automatically generating the grammatical structure of biological sequences in an inference framework of string compression algorithms. Our original motivation was to find any grammatical traits of several cancer genes that can be detected by string compression algorithms. Through this research, we could not find any meaningful unique traits of the cancer genes yet, but we could observe some interesting traits in regards to the relationship among gene length, similarity of sequences, the patterns of the generated grammar, and compression rate.

A Novel Scalable and Storage-Efficient Architecture for High Speed Exact String Matching

  • Peiravi, Ali;Rahimzadeh, Mohammad Javad
    • ETRI Journal
    • /
    • v.31 no.5
    • /
    • pp.545-553
    • /
    • 2009
  • String matching is a fundamental element of an important category of modern packet processing applications which involve scanning the content flowing through a network for thousands of strings at the line rate. To keep pace with high network speeds, specialized hardware-based solutions are needed which should be efficient enough to maintain scalability in terms of speed and the number of strings. In this paper, a novel architecture based upon a recently proposed data structure called the Bloomier filter is proposed which can successfully support scalability. The Bloomier filter is a compact data structure for encoding arbitrary functions, and it supports approximate evaluation queries. By eliminating the Bloomier filter's false positives in a space efficient way, a simple yet powerful exact string matching architecture is proposed that can handle several thousand strings at high rates and is amenable to on-chip realization. The proposed scheme is implemented in reconfigurable hardware and we compare it with existing solutions. The results show that the proposed approach achieves better performance compared to other existing architectures measured in terms of throughput per logic cells per character as a metric.