• Title/Summary/Keyword: text vector

Search Result 283, Processing Time 0.025 seconds

Inverted Index based Modified Version of KNN for Text Categorization

  • Jo, Tae-Ho
    • Journal of Information Processing Systems
    • /
    • v.4 no.1
    • /
    • pp.17-26
    • /
    • 2008
  • This research proposes a new strategy where documents are encoded into string vectors and modified version of KNN to be adaptable to string vectors for text categorization. Traditionally, when KNN are used for pattern classification, raw data should be encoded into numerical vectors. This encoding may be difficult, depending on a given application area of pattern classification. For example, in text categorization, encoding full texts given as raw data into numerical vectors leads to two main problems: huge dimensionality and sparse distribution. In this research, we encode full texts into string vectors, and modify the supervised learning algorithms adaptable to string vectors for text categorization.

Automation of Expert Classification in Knowledge Management Systems Using Text Categorization Technique (문서 범주화를 이용한 지식관리시스템에서의 전문가 분류 자동화)

  • Yang, Kun-Woo;Huh, Soon-Young
    • Asia pacific journal of information systems
    • /
    • v.14 no.2
    • /
    • pp.115-130
    • /
    • 2004
  • This paper proposes how to build an expert profile database in KMS, which provides the information of expertise that each expert possesses in the organization. To manage tacit knowledge in a knowledge management system, recent researches in this field have shown that it is more applicable in many ways to provide expert search mechanisms in KMS to pinpoint experts in the organizations with searched expertise so that users can contact them for help. In this paper, we develop a framework to automate expert classification using a text categorization technique called Vector Space Model, through which an expert database composed of all the compiled profile information is built. This approach minimizes the maintenance cost of manual expert profiling while eliminating the possibility of incorrectness and obsolescence resulted from subjective manual processing. Also, we define the structure of expertise so that we can implement the expert classification framework to build an expert database in KMS. The developed prototype system, "Knowledge Portal for Researchers in Science and Technology," is introduced to show the applicability of the proposed framework.

Scene Text Detection Using Color-Based Binarization and Text Region Verification Using Support Vector Machine (색기반 이진화를 이용한 장면 텍스트 추출과 써포트 벡터머신을 이용한 텍스트 영역 검증)

  • Jang, Dae-Geun;Kim, Eui-Jeong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2007.06a
    • /
    • pp.161-163
    • /
    • 2007
  • 기존의 텍스트 추출을 위한 이진화 방법은 입력 이미지를 명도 이미지로 변환한 뒤 이진화 하는 방법을 사용하였다. 이러한 방법은 칼라 이미지에서는 극명히 구분되는 색이라 할지라도 명도 이미지로 변환하는 과정에서 같은 밝기를 같게 되는 경우(예를 들어, 배경은 붉은색, 텍스트는 초록색), 텍스트를 추출하는 데 어려움이 있다. 본 논문에서는 이러한 문제를 해결하기 위해 입력 이미지를 R, G, B로 분리하고 각각을 이진화 하여 텍스트를 추출하고 다해상도 웨이블릿(Wavelet) 변환을 이용하여 텍스트의 획 특징을 추출하여 추출된 특징들을 SVM(Support Vector Machine) 분류기로 검증하여 최종 텍스트 영역을 확정한다. 제안한 방법을 적용함으로써 명도 정보만으로는 추출하기 어려웠던 텍스트 영역을 효과적으로 추출하고 텍스트와 구별하기 어려운 영역을 획수준으로 검증할 수 있었다.

  • PDF

A Novel Text to Image Conversion Method Using Word2Vec and Generative Adversarial Networks

  • LIU, XINRUI;Joe, Inwhee
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2019.05a
    • /
    • pp.401-403
    • /
    • 2019
  • In this paper, we propose a generative adversarial networks (GAN) based text-to-image generating method. In many natural language processing tasks, which word expressions are determined by their term frequency -inverse document frequency scores. Word2Vec is a type of neural network model that, in the case of an unlabeled corpus, produces a vector that expresses semantics for words in the corpus and an image is generated by GAN training according to the obtained vector. Thanks to the understanding of the word we can generate higher and more realistic images. Our GAN structure is based on deep convolution neural networks and pixel recurrent neural networks. Comparing the generated image with the real image, we get about 88% similarity on the Oxford-102 flowers dataset.

VoIP-Based Voice Secure Telecommunication Using Speaker Authentication in Telematics Environments (텔레매틱스 환경에서 화자인증을 이용한 VoIP기반 음성 보안통신)

  • Kim, Hyoung-Gook;Shin, Dong
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.10 no.1
    • /
    • pp.84-90
    • /
    • 2011
  • In this paper, a VoIP-based voice secure telecommunication technology using the text-independent speaker authentication in the telematics environments is proposed. For the secure telecommunication, the sender's voice packets are encrypted by the public-key generated from the speaker's voice information and submitted to the receiver. It is constructed to resist against the man-in-the middle attack. At the receiver side, voice features extracted from the received voice packets are compared with the reference voice-key received from the sender side for the speaker authentication. To improve the accuracy of text-independent speaker authentication, Gaussian Mixture Model(GMM)-supervectors are applied to Support Vector Machine (SVM) kernel using Bayesian information criterion (BIC) and Mahalanobis distance (MD).

On Statistical Estimation of Multivariate (Vector-valued) Process Capability Indices with Bootstraps)

  • Cho, Joong-Jae;Park, Byoung-Sun;Lim, Soo-Duck
    • Communications for Statistical Applications and Methods
    • /
    • v.8 no.3
    • /
    • pp.697-709
    • /
    • 2001
  • In this paper we study two vector-valued process capability indices $C_{p}$=($C_{px}$, $C_{py}$ ) and C/aub pm/=( $C_{pmx}$, $C_{pmy}$) considering process capability indices $C_{p}$ and $C_{pm}$ . First, two asymptotic distributions of plug-in estimators $C_{p}$=($C_{px}$, $C_{py}$ ) and $C_{pm}$ =) $C_{pmx}$, $C_{pmy}$) are derived.. With the asymptotic distributions, we propose asymptotic confidence regions for our indices. Next, obtaining the asymptotic distributions of two bootstrap estimators $C_{p}$=($C_{px}$, $C_{py}$ )and $C_{pm}$ =( $C_{pmx}$, $C_{pmy}$) with our bootstrap algorithm, we will provide the consistency of our bootstrap for statistical inference. Also, with the consistency of our bootstrap, we propose bootstrap asymptotic confidence regions for our indices. (no abstract, see full-text)see full-text)e full-text)

  • PDF

Identifying Mobile Owner based on Authorship Attribution using WhatsApp Conversation

  • Almezaini, Badr Mohammd;Khan, Muhammad Asif
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.7
    • /
    • pp.317-323
    • /
    • 2021
  • Social media is increasingly becoming a part of our daily life for communicating each other. There are various tools and applications for communication and therefore, identity theft is a common issue among users of such application. A new style of identity theft occurs when cybercriminals break into WhatsApp account, pretend as real friends and demand money or blackmail emotionally. In order to prevent from such issues, data mining can be used for text classification (TC) in analysis authorship attribution (AA) to recognize original sender of the message. Arabic is one of the most spoken languages around the world with different variants. In this research, we built a machine learning model for mining and analyzing the Arabic messages to identify the author of the messages in Saudi dialect. Many points would be addressed regarding authorship attribution mining and analysis: collect Arabic messages in the Saudi dialect, filtration of the messages' tokens. The classification would use a cross-validation technique and different machine-learning algorithms (Naïve Baye, Support Vector Machine). Results of average accuracy for Naïve Baye and Support Vector Machine have been presented and suggestions for future work have been presented.

Speaker Verification Using SVM Kernel with GMM-Supervector Based on the Mahalanobis Distance (Mahalanobis 거리측정 방법 기반의 GMM-Supervector SVM 커널을 이용한 화자인증 방법)

  • Kim, Hyoung-Gook;Shin, Dong
    • The Journal of the Acoustical Society of Korea
    • /
    • v.29 no.3
    • /
    • pp.216-221
    • /
    • 2010
  • In this paper, we propose speaker verification method using Support Vector Machine (SVM) kernel with Gaussian Mixture Model (GMM)-supervector based on the Mahalanobis distance. The proposed GMM-supervector SVM kernel method is combined GMM with SVM. The GMM-supervectors are generated by GMM parameters of speaker and other speaker utterances. A speaker verification threshold of GMM-supervectors is decided by SVM kernel based on Mahalanobis distance to improve speaker verification accuracy. The experimental results for text-independent speaker verification using 20 speakers demonstrates the performance of the proposed method compared to GMM, SVM, GMM-supervector SVM kernel based on Kullback-Leibler (KL) divergence, and GMM-supervector SVM kernel based on Bhattacharyya distance.

Text Summarization on Large-scale Vietnamese Datasets

  • Ti-Hon, Nguyen;Thanh-Nghi, Do
    • Journal of information and communication convergence engineering
    • /
    • v.20 no.4
    • /
    • pp.309-316
    • /
    • 2022
  • This investigation is aimed at automatic text summarization on large-scale Vietnamese datasets. Vietnamese articles were collected from newspaper websites and plain text was extracted to build the dataset, that included 1,101,101 documents. Next, a new single-document extractive text summarization model was proposed to evaluate this dataset. In this summary model, the k-means algorithm is used to cluster the sentences of the input document using different text representations, such as BoW (bag-of-words), TF-IDF (term frequency - inverse document frequency), Word2Vec (Word-to-vector), Glove, and FastText. The summary algorithm then uses the trained k-means model to rank the candidate sentences and create a summary with the highest-ranked sentences. The empirical results of the F1-score achieved 51.91% ROUGE-1, 18.77% ROUGE-2 and 29.72% ROUGE-L, compared to 52.33% ROUGE-1, 16.17% ROUGE-2, and 33.09% ROUGE-L performed using a competitive abstractive model. The advantage of the proposed model is that it can perform well with O(n,k,p) = O(n(k+2/p)) + O(nlog2n) + O(np) + O(nk2) + O(k) time complexity.

Combining Multiple Classifiers for Automatic Classification of Email Documents (전자우편 문서의 자동분류를 위한 다중 분류기 결합)

  • Lee, Jae-Haeng;Cho, Sung-Bae
    • Journal of KIISE:Software and Applications
    • /
    • v.29 no.3
    • /
    • pp.192-201
    • /
    • 2002
  • Automated text classification is considered as an important method to manage and process a huge amount of documents in digital forms that are widespread and continuously increasing. Recently, text classification has been addressed with machine learning technologies such as k-nearest neighbor, decision tree, support vector machine and neural networks. However, only few investigations in text classification are studied on real problems but on well-organized text corpus, and do not show their usefulness. This paper proposes and analyzes text classification methods for a real application, email document classification task. First, we propose a combining method of multiple neural networks that improves the performance through the combinations with maximum and neural networks. Second, we present another strategy of combining multiple machine learning classifiers. Voting, Borda count and neural networks improve the overall classification performance. Experimental results show the usefulness of the proposed methods for a real application domain, yielding more than 90% precision rates.