• Proceedings of the IEEK Conference
• /
• 2003.11b
• /
• pp.145-148
• /
• 2003

The agglutinative nature of Korean language makes the problem of automatic indexing of Korean much different from that of Indo-Eroupean languages. Especially, indexing with compound nouns in Korean is very problematic because of the exponential number of possible analysis and the existence of unknown words. To deal with this compound noun indexing problem, we propose a new indexing methods which combines the merits of the morpheme-based indexing methods and the n-gram based indexing methods. Through the experiments, we also find that the best performance of n-gram indexing methods can be achieved with 1.75-gram which is never considered in the previous researches.

• Bioinformatics and Biosystems
• /
• v.1 no.1
• /
• pp.46-50
• /
• 2006

According to the advancement of experimental techniques in molecular biology, genomic and protein sequence databases are increasing in size exponentially, and mean sequence lengths are also increasing. Because the sizes of these databases become larger, it is difficult to search similar sequences in biological databases with significant homologies to a query sequence. In this paper, we present the N-gram indexing method to retrieve similar sequences fast, precisely and comparably. This method regards a protein sequence as a text written in language of 20 amino acid codes, adapts N-gram tokens of fixed-length as its indexing scheme for sequence strings. After such tokens are indexed for all the sequences in the database, sequences can be searched with information retrieval algorithms. Using this new method, we have developed a protein sequence search system named as ProSeS (PROtein Sequence Search). ProSeS is a protein sequence analysis system which provides overall analysis results such as similar sequences with significant homologies, predicted subcellular locations of the query sequence, and major keywords extracted from annotations of similar sequences. We show experimentally that the N-gram indexing approach saves the retrieval time significantly, and that it is as accurate as current popular search tool BLAST.

• MALSORI
• /
• no.67
• /
• pp.149-166
• /
• 2008

In spoken document retrievals (SDR), subword (typically phonemes) indexing term is used to avoid the out-of-vocabulary (OOV) problem. It makes the indexing and retrieval process independent from any vocabulary. It also requires a small corpus to train the acoustic model. However, subword indexing term approach has a major drawback. It shows higher word error rates than the large vocabulary continuous speech recognition (LVCSR) system. In this paper, we propose an probabilistic slot detection and n-gram based string matching method for phone based spoken document retrievals to overcome high error rates of phone recognizer. Experimental results have shown 9.25% relative improvement in the mean average precision (mAP) with 1.7 times speed up in comparison with the baseline system.

• Journal of the Korean Society for information Management
• /
• v.13 no.1
• /
• pp.47-63
• /
• 1996

Conventional automatic indexing methods for Hangul texts can be classified into two groups as follows: One is to extract index terms by removing non-indexable segments from word-phrases, and the other is to generate index terms from the morphemes of word-phrases. The former suffers from the problem of word boundaries when documents contain many compound nouns. The latter can overcome the word boundary problem by extracting simple nouns, but has many overheads to develop a lot of linguistic knowledges needed in the indexing procedure. In this paper we propose a new indexing method based on n-grams. This method alleviates the problems of previous indexing methods related with word boundaries and linguistic knowledges. We also compare the effectiveness of the n-gram based indexing method with that of the previous ones.

• Proceedings of the Korea Society of Information Technology Applications Conference
• /
• 2002.11a
• /
• pp.412-416
• /
• 2002

멀티미디어 검색 시스템들은 아직까지 내용 기발에 의한 검색기술이 실용적으로 쓰일 만큼 높은 성능을 보이고 있지 않기 때문에 텍스트에 의한 검색만을 지원하고 있는 실정이다. HTML 문서에 나타나는 텍스트 중 이미지 아래에 붙은 표제나 이미지 링크에 붙어 있는 텍스트를 골라내어 이미지의 색인 정보로 이용하여 텍스트를 추출하는 기법을 제안하였다. 텍스트를 추출하기 위해 N-Gram 색인 방법을 사용하였으며 한편 검색 효율을 높이기 위해서 질의 의도가 큰 단어에 가중치를 부여하였다.

• Proceedings of the Korea Society for Industrial Systems Conference
• /
• 2002.11a
• /
• pp.412-416
• /
• 2002

멀티미디어 검색 시스템들은 아직까지 내용 기반에 의한 검색기술이 실용적으로 쓰일 만큼 높은 성능을 보이고 있지 않기 때문에 텍스트에 의한 검색만을 지원하고 있는 실정이다. HTML 문서에 나타나는 텍스트 중 이미지 아래에 붙은 표제나 이미지 링크에 붙어 있는 텍스트를 골라내어 이미지의 색인 정보로 이용하여 텍스트를 추출하는 기법을 제안하였다. 텍스트를 추출하기 위해 N-Gram 색인 방법을 사용하였으며 한편 검색 효율을 높이기 위해서 질의 의도가 큰 단어에 가중치를 부여하였다.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.14 no.2
• /
• pp.23-33
• /
• 2004

Because of a rapid growth of internet environment, it is also fast increasing to exchange message using e-mail. But, despite the convenience of e-mail, it is rising a currently bi9 issue to waste their time and cost due to the spam mail in an individual or enterprise. Many kinds of solutions have been studied to solve harmful effects of spam mail. Such typical methods are as follows; pattern matching using the keyword with representative method and method using the probability like Naive Bayesian. In this paper, we propose a classification method of spam mails from normal mails using Support Vector Machine, which has excellent performance in pattern classification problems, to compensate for the problems of existing research. Especially, the proposed method practices efficiently a teaming procedure with a word dictionary including a generated index by the n-Gram. In the conclusion, we verified the proposed method through the accuracy comparison of spm mail separation between an existing research and proposed scheme.

• Journal of the Korean Society for information Management
• /
• v.30 no.3
• /
• pp.23-48
• /
• 2013

This study compared and analyzed 15 CBMR (Content-based Music Retrieval) systems accessible on the web in terms of DB size and type, query type, access point, input and output type, and search functions, with reviewing features of music information and techniques used for transforming or transcribing of music sources, extracting and segmenting melodies, extracting and indexing features of music, and matching algorithms for CBMR systems. Application of text information retrieval techniques such as inverted indexing, N-gram indexing, Boolean search, truncation, keyword and phrase search, normalization, filtering, browsing, exact matching, similarity measure using edit distance, sorting, etc. to enhancing the CBMR; effort for increasing DB size and usability; and problems in extracting melodies, deleting stop notes in queries, and using solfege as pitch information were found as the results of analysis.

• Journal of KIISE:Databases
• /
• v.34 no.4
• /
• pp.283-303
• /
• 2007

Recently, the study of efficient way to store and retrieve enormous music data is becoming the one of important issues in the multimedia database. Most general method of MIR (Music Information Retrieval) includes a text-based approach using text information to search a desired music. However, if users did not remember the keyword about the music, it can not give them correct answers. Moreover, since these types of systems are implemented only for exact matching between the query and music data, it can not mine any information on similar music data. Thus, these systems are inappropriate to achieve similarity matching of music data. In order to solve the problem, we propose an Efficient Query-By-Humming System (EQBHS) with a content-based indexing method that efficiently retrieve and store music when a user inquires with his incorrect humming. For the purpose of accelerating query processing in EQBHS, we design indices for significant melodies, which are 1) frequent melodies occurring many times in a single music, on the assumption that users are to hum what they can easily remember and 2) melodies partitioned by rests. In addition, we propose an error tolerated mapping method from a note to a character to make searching efficient, and the frequent melody extraction algorithm. We verified the assumption for frequent melodies by making up questions and compared the performance of the proposed EQBHS with N-gram by executing various experiments with a number of music data.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.18 no.6A
• /
• pp.129-137
• /
• 2008

As the use of the internet increases, the distribution of spam mail has also vastly increased. The email's main use was for the exchange of information, however, currently it is being more frequently used for advertisement and malware distribution. This is a serious problem because it consumes a large amount of the limited internet resources. Furthermore, an extensive amount of computer, network and human resources are consumed to prevent it. As a result much research is being done to prevent and filter spam. Currently, research is being done on readable sentences which do not use proper grammar. This type of spam can not be classified by previous vocabulary analysis or document classification methods. This paper proposes a method to filter spam by using the subject of the mail and N-GRAM for indexing and Bayesian, SVM algorithms for classification.