• Journal of KIISE:Software and Applications
• /
• v.36 no.1
• /
• pp.62-69
• /
• 2009

The purpose of document summarization is to provide easy and quick understanding of documents by extracting summarized information from the documents produced by various application programs. In this paper, we propose a document summarization method that creates and analyzes a connection graph representing the similarity of keyword lists of sentences in a document taking into account the mean length(the number of keywords) of sentences of the document. We implemented a system that automatically generate a summary from a document using the proposed method. To evaluate the performance of the method, we used a set of 20 documents associated with their correct summaries and measured the precision, the recall and the F-measure. The experiment results show that the proposed method is more efficient compared with the existing methods.

• Journal of the Korean Data and Information Science Society
• /
• v.14 no.4
• /
• pp.853-861
• /
• 2003

An improved K-means document clustering method has been presented, where a concept vector is manipulated for each cluster on the basis of cosine similarity of text documents. The concept vectors are unit vectors that have been normalized on the n-dimensional sphere. Because the standard K-means method is sensitive to initial starting condition, our improvement focused on starting condition for estimating the modes of a distribution. The improved K-means clustering algorithm has been applied to a set of text documents, called Classic3, to test and prove efficiency and correctness of clustering result, and showed 7% improvements in its worst case.

• Journal of Software Assessment and Valuation
• /
• v.15 no.2
• /
• pp.1-7
• /
• 2019

Program similarity analysis consists of substantial similarity and access. Substantial similarity is a judgment of how similarly the program source code is quantitatively. Access determines the degree of similarity by analyzing comments in the program or other contextual evidence. In the case of manuals, it may be the subject of legitimacy analysis. Manuals can be divided into three types as follows. First, a master manual is a document created during the development stage of a product. It is a user manual that contains all the functionality of the product and its derivatives. Second, the customer manual is a manual that is open only to the primary customer and orderer. Third, the user manual is a document that is applied to the final OEM production stage and is open to the end purchaser. In this paper, we compare the master manual seized from the suspect and the master manual provided by the suspect on the Internet. It then determines how similar this master manual is and includes the victim company's original and property values.

• Journal of KIISE
• /
• v.41 no.9
• /
• pp.728-743
• /
• 2014

Finger printing and sequence alignment are well-known approaches for document similarity comparison. A fingerprinting method is simple and fast, but it can not find particular similar regions. A string alignment method is used for identifying regions of similarity by arranging the sequences of a string. It has an advantage of finding particular similar regions, but it also has a disadvantage of taking more computing time. The Multi-Level Alignment (MLA) is a new method designed for taking the advantages of both methods. The MLA divides input documents into uniform length blocks, and then extracts fingerprints from each block and calculates similarity of block pairs by comparing the fingerprints. A similarity table is created in this process. Finally, sequence alignment is used for specifying longest similar regions in the similarity table. The MLA allows users to change block's size to control proportion of the fingerprint algorithm and the sequence alignment. As a document is divided into several blocks, similar regions are also fragmented into two or more blocks. To solve this fragmentation problem, we proposed a united block method. Experimentally, we show that computing document's similarity with the united block is more accurate than the original MLA method, with minor time loss.

• Journal of Digital Contents Society
• /
• v.12 no.4
• /
• pp.457-465
• /
• 2011

The annual R&D investment in Korea increased by an average of 12.2percent during the last 5 years. Therefore, prevention of duplicate projects being performed became an important factor in promoting the efficiency of R&D investment and the originality of R&D projects. On measuring the similarity of projects, the measurement model used to estimate the accuracy of the similarity is crucial. In this paper, we propose an advanced measurement model on checking the similarity of R&D projects for promoting the efficiency of R&D investment. The proposed model is made up of the following steps for the model measurement, sampling and analyzing. During the sampling step, we append the abstract of R&D reports on the search engine based on document vector. We then measure the similarity on projects to use research title network which is consists of the compound keyword and the weight of items on during the analysis. The proposed method improved the accuracy for measuring the similarity of projects by an average of 0.19 over the existing search engine and by 9.25 over the simple keyword search on R&D projects. On searching the similarity with the appending conditions and high sampling, it improved the accuracy of measuring the similarity of R&D projects.

• KSCE Journal of Civil and Environmental Engineering Research
• /
• v.32 no.1D
• /
• pp.51-59
• /
• 2012

An efficient application method of XML schema matching technique to the document structure of structural calculation document (SCD) of bridge is proposed. With 30 case studies, a parametric study on weightings of name, sibling, child, and parent elements of XML scheme component that are used in the similarity measure of XML schema matching technique has been performed, and suitable weighting to analyze document structure of SCD is suggested. A simplified formula for quantification of similarity is also introduced to reduce computation time in huge scale document structure of SCDs. Numerical experiments show that the suggested method can increase the accuracy of XML schema matching by 10% with suitable weighting parameters, and can maintain almost the same accuracy without weighting parameters compared to previous studies. In addition, computation time can be reduced dramatically when the proposed simplified formula for the quantification of similarity is used. In the numerical experiments of testing 20 practical SCDs of bridges, the suggested method is superior to previous studies in the accuracy of analyzing document structure and 4 to 460 times faster than the previous results in computation time.

• The Journal of Society for e-Business Studies
• /
• v.25 no.3
• /
• pp.77-93
• /
• 2020

The literature has reported that hierarchical classification methods generally outperform the flat classification methods for a multi-class document classification problem. Unlike the literature that has constructed a class hierarchy, this paper evaluates the performance of hierarchical and flat classification methods under a situation where the class hierarchy is predefined. We conducted numerical evaluations for two data sets; research papers on climate change adaptation technologies in water sector and 20NewsGroup open data set. The evaluation results show that the hierarchical classification method outperforms the flat classification methods under a certain condition, which differs from the literature. The performance of hierarchical classification method over flat classification method depends on class similarities at levels in the class structure. More importantly, the hierarchical classification method works better when the upper level similarity is less that the lower level similarity.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.13 no.12
• /
• pp.2603-2608
• /
• 2009

Document clustering is an important method for document analysis and is used in many different information retrieval applications. This paper proposes a new document clustering model using the clustering method based NMF(non-negative matrix factorization) and refinement of documents in cluster by using coherence of cluster. The proposed method can improve the quality of document clustering because the re-assigned documents in cluster by using coherence of cluster based similarity between documents, the semantic feature matrix and the semantic variable matrix, which is used in document clustering, can represent an inherent structure of document set more well. The experimental results demonstrate appling the proposed method to document clustering methods achieves better performance than documents clustering methods.

• Journal of KIISE:Software and Applications
• /
• v.29 no.6
• /
• pp.367-376
• /
• 2002

XML allows users to define elements using arbitrary words and organize them in a nested structure. These features of XML offer both challenges and opportunities in information retrieval and document management. In this paper, we propose a new methodology for computing similarity considering XML semantics - meanings of the elements and nested structures of XML documents. We generate extended-element vectors, using thesaurus, to normalize synonyms, compound words, and abbreviations and build similarity matrix using them. And then we compute similarity between XML elements. We also discover and minimize XML structure using automata(NFA(Nondeterministic Finite Automata) and DFA(Deterministic Finite automata). We compute similarity between XML structures using similarity matrix between elements and minimized XML structures. Our methodology considering XML semantics shows 100% accuracy in identifying the category of real documents from on-line bookstore.

• Journal of the Korean Society for information Management
• /
• v.24 no.1 s.63
• /
• pp.251-271
• /
• 2007

This paper studies the problem of classifying documents with labeled and unlabeled learning data, especially with regards to using document similarity features. The problem of using unlabeled data is practically important because in many information systems obtaining training labels is expensive, while large quantities of unlabeled documents are readily available. There are two steps In general semi-supervised learning algorithm. First, it trains a classifier using the available labeled documents, and classifies the unlabeled documents. Then, it trains a new classifier using all the training documents which were labeled either manually or automatically. We suggested two types of semi-supervised learning algorithm with regards to using document similarity features. The one is one step semi-supervised learning which is using unlabeled documents only to generate document similarity features. And the other is two step semi-supervised learning which is using unlabeled documents as learning examples as well as similarity features. Experimental results, obtained using support vector machines and naive Bayes classifier, show that we can get improved performance with small labeled and large unlabeled documents then the performance of supervised learning which uses labeled-only data. When considering the efficiency of a classifier system, the one step semi-supervised learning algorithm which is suggested in this study could be a good solution for improving classification performance with unlabeled documents.