• Journal of Computing Science and Engineering
• /
• 제3권1호
• /
• pp.15-26
• /
• 2009

Given an n-length text T over a $\sigma$-size alphabet, we present a compressed representation of T which supports retrieving queries of rank/select/access and updating queries of insert/delete. For a measure of compression, we use the empirical entropy H(T), which defines a lower bound nH(T) bits for any algorithm to compress T of n log $\sigma$ bits. Our representation takes this entropy bound of T, i.e., nH(T) $\leq$ n log $\sigma$ bits, and an additional bits less than the text size, i.e., o(n log $\sigma$) + O(n) bits. In compressed space of nH(T) + o(n log $\sigma$) + O(n) bits, our representation supports O(log n) time queries for a log n-size alphabet and its extension provides O(($1+\frac{{\log}\;{\sigma}}{{\log}\;{\log}\;n}$) log n) time queries for a $\sigma$-size alphabet.

• Journal of information and communication convergence engineering
• /
• 제8권4호
• /
• pp.377-382
• /
• 2010

With the advent of flash memory based new storage device (SSD), there is considerable interest within the computer industry in using flash memory based storage devices for many different types of application. The dynamic index structure of large text collections has been a primary issue in the Information Retrieval Applications among them. Previous studies have proven the three approaches to be effective: In- Place, merge-based index structure and a combination of both. The above-mentioned strategies have been researched with the traditional storage device (HDD) which has a constraint on how keep the contiguity of dynamic data. However, in case of the new storage device, we don' have any constraint contiguity problems due to its low access latency time. But, although the new storage device has superiority such as low access latency and improved I/O throughput speeds, it is still not well suited for traditional dynamic index structures because of the poor random write throughput in practical systems. Therefore, using the experimental performance evaluation of various index maintenance schemes on the new storage device, we propose an efficient index structure for new storage device that improves significantly the index maintenance speed without degradation of query performance.

• 인지과학
• /
• 제4권1호
• /
• pp.213-242
• /
• 1993

그동안 자동색인 기법에 대해서는 단일어 색인,구색인,시소러스 기반 색인 등의 연구가 이루어져 왔는데,단일어 색인 기법이 단일어에 기초한 단순한 색인 방법 임에도 불구하고 다른 두가지 기법보다 일반적으로 우수하다고 알려져왔다. 시소러스 기반 색인은 이중에서도 검색효율이 낮은 것으로 알려져 왔는데,이는 일반적으로 시소러스가 포함하고 있는 색인용어들이 한정되어 있어 색인하려는 자료들이 이색인 용어에 부합(match)되지 않을 경우 색인 자체가 이루어 지지 않기 때문이다. 본 연구에서는 시소러스 기반 색인이 지금까지 기법으로는 검색효율이 좋지 않지만 실제 전문 색인들이 하는 색인과 매우 유사하다는 장범에 기초하여,입력 자료를 구문분석하고,분석된 자료들과 색인용어들을 정확부합(exact match)이 아닌 부분부합(partial match)을 통하여 색인 함으로써 검색효율이 우수한 시소러스기반 자동 색인 시스템을 개발하고자 한다. 본 연구에서 개발된 색인 시스템이 THINS는 우선 시소러스를 트리형태로 구성하고 입력자료들을 KAIST에서 개발한 언어번역기 MATES/EK를 통하여 구문분석한 후 명사구들만 뽑아낸다.그다음 명사구에 있는 용어들중 불용어를 제거하고 스테밍작업을 진행한후 생기는 형태를 색인 용어들과 부분부합 과정을 반복하여,유사한 색인 용어들과 가능하면 색인이 되도록 한다. 본 연구에서는 CACM 데이타 집합을 가지고 본 시소러스 기반 색인 시스템과 단일어 색인방식을 혼성지식기반 시스템인 HYKIS에서 성능을 평가하였다.이 성능평가에서 시소러스를 기반으로 하는 색인 시스템이 단일어 색인방식보다 회상도에서는 8-9%떨어지지만 정확도에서는 10%정도 높은 결과를 나타내었다.그러나 이는 기존의 시소러스 기반시스템이 단일어 색인 방식보다 정확도가 25%-30%정도 떨어진다는 것을 비추어 볼때 기존의 방식보다 우수한 것이라 평가된다.또한 CACM 에서 제공하는 시소러스인 CRCS 가 1000여개의 색인어밖에 포함하고 있어 매우 불완전한 것이라는 것을 고려하면,THINS가 최근에 개발되고 있는 시소러스와 접속된다면 매우 우수한 검색효율을 내리라 사료된다.

• 한국정보처리학회논문지
• /
• 제5권5호
• /
• pp.1109-1118
• /
• 1998

최근에 비디오 데이터베이스에 대하여 관심이 집중되고 있다. 비디오 문서도 텍스트 문서와 마찬가지로 계층적 논리 구조를 포함하고 있다. 사용자가 구조 질의로 이 구조를 활용한다면, 내용 질의에만 의존하는 것보다 더 큰 효과를 거둘 수가 있다. 구조 질의를 효율적으로 처리하기 위해서는 비디오의 문서 요소에 빨리 접근 할 수 있도록 지원하는 인덱스 구조가 필수적이다. 이 논문에서는 트리 구조의 비디오 문서 모형을 제시하고 비디오 문서를 위한 새로운 역 인덱스 구조를 제안한다. 그리고 이 인덱스 구조의 저장 장소 요구량과 디스크 접근 시간을 평가하고, 분석 결과를 제시한다.

• 음성과학
• /
• 제2권
• /
• pp.159-172
• /
• 1997

This paper proposes a prosodic representation system of the Kyungnam dialect of Korean, based on the ToBI system. In this system, diverse intonation patterns are transcribed on the four parallel tiers: a tone tier, a break index tier, an orthographic tier, and a miscellaneous tier. The tone tier employs pitch accents, phrase accents, and boundary tones marked with diacritics in order to represent various pitch events. The break index tier uses five break indices, numbered from 0 to 4, in order to represent degrees of connectiveness in speech by associating each inter-word position with a break index. In this, each break index represents a boundary of some kind of constituent. This system can contribute not only to a more detailed theory connecting prosody, syntax, and intonation, but also to current text-to-speech synthesis approaches, speech recognition, and other quantitative computational modellings.

• 대한전기학회:학술대회논문집
• /
• 대한전기학회 1994년도 하계학술대회 논문집 C
• /
• pp.1428-1430
• /
• 1994

The accumulation of space charge in PE/PE/PE laminates was investigated using the low density polyethylenes having different melt indices from 0.25 to 45 g/10 min. Both heterocharge and homocharge accumulated in parent polyethylenes depend on the value of melt index. A general rule found in this study was that the heterocharge decreases as the melt index increases and finally the homocharge accumulates when the melt index is very high. Because of this reason, PE laminates showed different charge formation characteristics depending on melt index of polyethylene. Details of up to date results are described in the text.

• 대한전자공학회:학술대회논문집
• /
• 대한전자공학회 2002년도 ITC-CSCC -1
• /
• pp.446-449
• /
• 2002

This paper represents a improved data compression algorithm of mixed text file by 2 byte completion Hangout and 1 byte alphabet from. Original LZW algorithm efficiently compress a alphabet text file but inefficiently compress a 2 byte completion Hangout text file. To solve this problem, data compression algorithm using 2 byte prefix field and 2 byte suffix field for compression table have developed. But it have a another problem that is compression ratio of alphabet text file decreased. In this paper, we proposes improved LZW algorithm, that is, compression table in the Extended LZW(ELZW) algorithm uses 2 byte prefix field for pointer of a table and 1 byte suffix field for repeat counter. where, a prefix field uses a pointer(index) of compression table and a suffix field uses a counter of overlapping or recursion text data in compression table. To increase compression ratio, after construction of compression table, table data are properly packed as different bit string in accordance with a alphabet, Hangout, and pointer respectively. Therefore, proposed ELZW algorithm is superior to 1 byte LZW algorithm as 7.0125 percent and superior to 2 byte LZW algorithm as 11.725 percent. This paper represents a improved data Compression algorithm of mixed text file by 2 byte completion Hangout and 1 byte alphabet form. This document is an example of what your camera-ready manuscript to ITC-CSCC 2002 should look like. Authors are asked to conform to the directions reported in this document.

• 한국멀티미디어학회논문지
• /
• 제1권2호
• /
• pp.162-172
• /
• 1998

본 논문에서는 영문 텍스트 검색 엔진인 MG(Managing Gigabytes) 시스템과 한글 형태소 분석기 HAM (Hangul Analysis Module)을 이용하여 기가바이트 크기의 텍스트 데이타 처리가 가능한 한글 텍스트 검색 엔진 HMG(Hangul MG)를 구현하였다. 한글 처리를 위해 KSC 5601 완성형 코드를 사용하여 데이타베이스 구축 단계와 질의 처리 단계에서 사용하였다. HMG의 개발을 위해 MG 시스템의 렉시칼 분석기와 파서, 인텍스 구성 모률을 수정하였다. HMG 시스템의 유용성을 보이기 위해 웹에서 한글 소설을 검색할 수 있도록 하는 N NOD (Novel On Demand) 시스템올 구현하였다. HMG 시스템은 한글이 포함된 대규모 전문 검색 시스템의 구축에 활용될 수 있다.

• 대한음성학회지:말소리
• /
• 제53호
• /
• pp.75-92
• /
• 2005

Prosody Generation Based on C-ToBI Representation for Text-to-SpeechSeungwon Kim, Yu Zheng, Gary Geunbae Lee, Byeongchang KimProsody modeling is critical in developing text-to-speech (TTS) systems where speech synthesis is used to automatically generate natural speech. In this paper, we present a prosody generation architecture based on Chinese Tone and Break Index (C-ToBI) representation. ToBI is a multi-tier representation system based on linguistic knowledge to transcribe events in an utterance. The TTS system which adopts ToBI as an intermediate representation is known to exhibit higher flexibility, modularity and domain/task portability compared with the direct prosody generation TTS systems. However, the cost of corpus preparation is very expensive for practical-level performance because the ToBI labeled corpus has been manually constructed by many prosody experts and normally requires a large amount of data for accurate statistical prosody modeling. This paper proposes a new method which transcribes the C-ToBI labels automatically in Chinese speech. We model Chinese prosody generation as a classification problem and apply conditional Maximum Entropy (ME) classification to this problem. We empirically verify the usefulness of various natural language and phonology features to make well-integrated features for ME framework.

• Journal of Information Processing Systems
• /
• 제16권2호
• /
• pp.277-300
• /
• 2020

This paper presents a method for clustering short text documents, such as news headlines, social media statuses, or instant messages. Due to the characteristics of these documents, which are usually short and sparse, an appropriate technique is required to discover hidden knowledge. The objective of this paper is to identify the combination of document representation, document distance, and document clustering that yields the best clustering quality. Document representations are expanded by external knowledge sources represented by a Distributed Representation. To cluster documents, a K-means partitioning-based clustering technique is applied, where the similarities of documents are measured by word mover's distance. To validate the effectiveness of the proposed method, experiments were conducted to compare the clustering quality against several leading methods. The proposed method produced clusters of documents that resulted in higher precision, recall, F1-score, and adjusted Rand index for both real-world and standard data sets. Furthermore, manual inspection of the clustering results was conducted to observe the efficacy of the proposed method. The topics of each document cluster are undoubtedly reflected by members in the cluster.