• Journal of the Korean Society for Library and Information Science
• /
• v.17
• /
• pp.339-366
• /
• 1989

Appearance of full-text online is the most encouraging phenomenon ·during the development of databases. The full-text databases of today is derived from by-product of electronic publication of printed materials. Now, there are also some movements toward electronic production of documents in Korea although not powerful. The present study is designed to examine the characteristics and effective retrieval method of full-text databases now commercially available through various vendors. The outline of this paper IS as follows: First, background and present situation of existing full-text database services through national and worldwide are examined. Second, free-text searching system of full-text databases is compared with controlled vocabulary system. The factors influencing on free-text retrieval performance, searching thesaurus, and hybrid or compromising system, which is using limited controlled vocabulary in conjunction with natural language for the enrichment needed for practical operation of the . system, are examined. Third, user demands through the analysis of preceding studies on 'various types of full-text databases are recognised. Fouth, application of CD-ROM full-text database to the libraries and information centers is examined as prospective resources for them. Finally, some problems and prospect of full-text databases are presented.

• Journal of Korean Society of Archives and Records Management
• /
• v.19 no.2
• /
• pp.147-172
• /
• 2019

With the provision of the original text information disclosure service, the time spent on determining the disclosure of the original text information decreased, and the number of original text information disclosure significantly increased. In public institutions, the risk of the exposure of personal information also increased. In this study, the status of personal information protection in the original text information disclosure service was investigated. Moreover, the causes of the exposure of personal information were analyzed, and improvements were proposed. The survey presented the following results. First, 13% of the original text information collected contains personal information, which is the nondisclosure information. Second, among the original text information that includes personal information, the original text information, including the personal information of the public official, was the most important. In particular, many records about vacation and medical leaves were found. Third, there were many cases in which information about the individual of the representative was exposed in the agency that deals mainly with the contract work. Fourth, a large volume of personal information was not detected by filtering personal information. Upon analyzing the cause of the exposure of personal information, the following improvements are suggested. First, privacy guidelines should be redesigned. Second, the person in charge of the task of deciding whether or not to disclose original text information should be trained further. Third, the excessive disclosure of information based on the government's quantitative performance should be eased. Fourth, the filtering function of the personal information of the original text information disclosure system should be improved.

• Journal of Information Processing Systems
• /
• v.4 no.2
• /
• pp.77-86
• /
• 2008

This research proposes a new neural network for text categorization which uses alternative representations of documents to numerical vectors. Since the proposed neural network is intended originally only for text categorization, it is called NTC (Neural Text Categorizer) in this research. Numerical vectors representing documents for tasks of text mining have inherently two main problems: huge dimensionality and sparse distribution. Although many various feature selection methods are developed to address the first problem, the reduced dimension remains still large. If the dimension is reduced excessively by a feature selection method, robustness of text categorization is degraded. Even if SVM (Support Vector Machine) is tolerable to huge dimensionality, it is not so to the second problem. The goal of this research is to address the two problems at same time by proposing a new representation of documents and a new neural network using the representation for its input vector.

• Journal of information and communication convergence engineering
• /
• v.18 no.1
• /
• pp.33-38
• /
• 2020

Elasticsearch is an open source search and analytics engine that can search petabytes of data in near real time. It is designed as a distributed system horizontally scalable and highly available. It provides RESTful APIs, thereby making it programming-language agnostic. Full text search of multilingual text requires language-specific analyzers and field mappings appropriate for indexing and searching multilingual text. Additionally, a language detector can be used in conjunction with the analyzers to improve the multilingual text search. Elasticsearch provides more than 40 language analysis plugins that can process text and extract language-specific tokens and language detector plugins that can determine the language of the given text. This study investigates three different approaches to index and search Chinese, Japanese, and Korean (CJK) text (single analyzer, multi-fields, and language detector-based), and identifies the advantages of the language detector-based approach compared to the other two.

• Journal of Information Processing Systems
• /
• v.13 no.4
• /
• pp.863-875
• /
• 2017

The traditional text similarity measurement methods based on word frequency vector ignore the semantic relationships between words, which has become the obstacle to text similarity calculation, together with the high-dimensionality and sparsity of document vector. To address the problems, the improved singular value decomposition is used to reduce dimensionality and remove noises of the text representation model. The optimal number of singular values is analyzed and the semantic relevance between words can be calculated in constructed semantic space. An inverted index construction algorithm and the similarity definitions between vectors are proposed to calculate the similarity between two documents on the semantic level. The experimental results on benchmark corpus demonstrate that the proposed method promotes the evaluation metrics of F-measure.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.12
• /
• pp.6080-6096
• /
• 2019

Text classification is one of the fundamental techniques in natural language processing. Numerous studies are based on text classification, such as news subject classification, question answering system classification, and movie review classification. Traditional text classification methods are used to extract features and then classify them. However, traditional methods are too complex to operate, and their accuracy is not sufficiently high. Recently, convolutional neural network (CNN) based one-hot method has been proposed in text classification to solve this problem. In this paper, we propose an improved method using CNN based skip-gram method for Chinese text classification and it conducts in Sogou news corpus. Experimental results indicate that CNN with the skip-gram model performs more efficiently than CNN-based one-hot method.

• Journal of Information Science Theory and Practice
• /
• v.10 no.spc
• /
• pp.86-95
• /
• 2022

Online recruitment websites discuss job demands in various fields, and job postings contain detailed job specifications. Analyzing this text can elucidate the features that determine job salaries. Text embedding models can learn the contextual information in a text, and explainable artificial intelligence frameworks can be used to examine in detail how text features contribute to the models' outputs. We collected 733,625 job postings using the WORKNET API and classified them into low, mid, and high-range salary groups. A text embedding model that predicts job salaries based on the text in job postings was trained with the collected data. Then, we applied the SHapley Additive exPlanations (SHAP) framework to the trained model and discovered the significant words that determine each salary class. Several limitations and remaining words are also discussed.

• Journal of information and communication convergence engineering
• /
• v.21 no.1
• /
• pp.9-16
• /
• 2023

The natural language on social network platforms has a certain front-to-back dependency in structure, and the direct conversion of Chinese text into a vector makes the dimensionality very high, thereby resulting in the low accuracy of existing text classification methods. To this end, this study establishes a deep learning model that combines a big data ultra-deep convolutional neural network (UDCNN) and long short-term memory network (LSTM). The deep structure of UDCNN is used to extract the features of text vector classification. The LSTM stores historical information to extract the context dependency of long texts, and word embedding is introduced to convert the text into low-dimensional vectors. Experiments are conducted on the social network platforms Sogou corpus and the University HowNet Chinese corpus. The research results show that compared with CNN + rand, LSTM, and other models, the neural network deep learning hybrid model can effectively improve the accuracy of text classification.

• Proceedings of the Korean Information Science Society Conference
• /
• 2006.10b
• /
• pp.233-237
• /
• 2006

This study proposes an innovative measure for evaluating the performance of text clustering. In using K-means algorithm and Kohonen Networks for text clustering, the number clusters is fixed initially by configuring it as their parameter, while in using single pass algorithm for text clustering, the number of clusters is not predictable. Using labeled documents, the result of text clustering using K-means algorithm or Kohonen Network is able to be evaluated by setting the number of clusters as the number of the given target categories, mapping each cluster to a target category, and using the evaluation measures of text. But in using single pass algorithm, if the number of clusters is different from the number of target categories, such measures are useless for evaluating the result of text clustering. This study proposes an evaluation measure of text clustering based on intra-cluster similarity and inter-cluster similarity, what is called CI (Clustering Index) in this article.

• Proceedings of the IEEK Conference
• /
• 2005.11a
• /
• pp.967-970
• /
• 2005

The Music Player MAF Player Format of ISO/IEC 23000-2 FDIS consists of MP3 data, MPEG-7 metadata and one optional JPEG image data based on MPEG-4 File Format. However, the current Music Player MAF format does not allow multiple JPEG image data or timed text data. It is helpful to use timed text data and multiple JPEG images in the various multimedia applications. For example, listening material for the foreign language needs an additional book which has text and images, the audio contents which can get image and text data can be helpful to understand the whole story and situations well. In this paper, we propose the detailed file structure in conjunction with MPEG-4 File Format in order to improve the functionalities, which carry multiple image data and text data with synchronization information between MP3 data and other resources.