• Title/Summary/Keyword: Label embedding

Search Result 18, Processing Time 0.021 seconds

Label Embedding for Improving Classification Accuracy UsingAutoEncoderwithSkip-Connections (다중 레이블 분류의 정확도 향상을 위한 스킵 연결 오토인코더 기반 레이블 임베딩 방법론)

  • Kim, Museong;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.27 no.3
    • /
    • pp.175-197
    • /
    • 2021
  • Recently, with the development of deep learning technology, research on unstructured data analysis is being actively conducted, and it is showing remarkable results in various fields such as classification, summary, and generation. Among various text analysis fields, text classification is the most widely used technology in academia and industry. Text classification includes binary class classification with one label among two classes, multi-class classification with one label among several classes, and multi-label classification with multiple labels among several classes. In particular, multi-label classification requires a different training method from binary class classification and multi-class classification because of the characteristic of having multiple labels. In addition, since the number of labels to be predicted increases as the number of labels and classes increases, there is a limitation in that performance improvement is difficult due to an increase in prediction difficulty. To overcome these limitations, (i) compressing the initially given high-dimensional label space into a low-dimensional latent label space, (ii) after performing training to predict the compressed label, (iii) restoring the predicted label to the high-dimensional original label space, research on label embedding is being actively conducted. Typical label embedding techniques include Principal Label Space Transformation (PLST), Multi-Label Classification via Boolean Matrix Decomposition (MLC-BMaD), and Bayesian Multi-Label Compressed Sensing (BML-CS). However, since these techniques consider only the linear relationship between labels or compress the labels by random transformation, it is difficult to understand the non-linear relationship between labels, so there is a limitation in that it is not possible to create a latent label space sufficiently containing the information of the original label. Recently, there have been increasing attempts to improve performance by applying deep learning technology to label embedding. Label embedding using an autoencoder, a deep learning model that is effective for data compression and restoration, is representative. However, the traditional autoencoder-based label embedding has a limitation in that a large amount of information loss occurs when compressing a high-dimensional label space having a myriad of classes into a low-dimensional latent label space. This can be found in the gradient loss problem that occurs in the backpropagation process of learning. To solve this problem, skip connection was devised, and by adding the input of the layer to the output to prevent gradient loss during backpropagation, efficient learning is possible even when the layer is deep. Skip connection is mainly used for image feature extraction in convolutional neural networks, but studies using skip connection in autoencoder or label embedding process are still lacking. Therefore, in this study, we propose an autoencoder-based label embedding methodology in which skip connections are added to each of the encoder and decoder to form a low-dimensional latent label space that reflects the information of the high-dimensional label space well. In addition, the proposed methodology was applied to actual paper keywords to derive the high-dimensional keyword label space and the low-dimensional latent label space. Using this, we conducted an experiment to predict the compressed keyword vector existing in the latent label space from the paper abstract and to evaluate the multi-label classification by restoring the predicted keyword vector back to the original label space. As a result, the accuracy, precision, recall, and F1 score used as performance indicators showed far superior performance in multi-label classification based on the proposed methodology compared to traditional multi-label classification methods. This can be seen that the low-dimensional latent label space derived through the proposed methodology well reflected the information of the high-dimensional label space, which ultimately led to the improvement of the performance of the multi-label classification itself. In addition, the utility of the proposed methodology was identified by comparing the performance of the proposed methodology according to the domain characteristics and the number of dimensions of the latent label space.

Constrained Sparse Concept Coding algorithm with application to image representation

  • Shu, Zhenqiu;Zhao, Chunxia;Huang, Pu
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.8 no.9
    • /
    • pp.3211-3230
    • /
    • 2014
  • Recently, sparse coding has achieved remarkable success in image representation tasks. In practice, the performance of clustering can be significantly improved if limited label information is incorporated into sparse coding. To this end, in this paper, a novel semi-supervised algorithm, called constrained sparse concept coding (CSCC), is proposed for image representation. CSCC considers limited label information into graph embedding as additional hard constraints, and hence obtains embedding results that are consistent with label information and manifold structure information of the original data. Therefore, CSCC can provide a sparse representation which explicitly utilizes the prior knowledge of the data to improve the discriminative power in clustering. Besides, a kernelized version of our proposed CSCC, namely kernel constrained sparse concept coding (KCSCC), is developed to deal with nonlinear data, which leads to more effective clustering performance. The experimental evaluations on the MNIST, PIE and Yale image sets show the effectiveness of our proposed algorithms.

An Efficient Deep Learning Ensemble Using a Distribution of Label Embedding

  • Park, Saerom
    • Journal of the Korea Society of Computer and Information
    • /
    • v.26 no.1
    • /
    • pp.27-35
    • /
    • 2021
  • In this paper, we propose a new stacking ensemble framework for deep learning models which reflects the distribution of label embeddings. Our ensemble framework consists of two phases: training the baseline deep learning classifier, and training the sub-classifiers based on the clustering results of label embeddings. Our framework aims to divide a multi-class classification problem into small sub-problems based on the clustering results. The clustering is conducted on the label embeddings obtained from the weight of the last layer of the baseline classifier. After clustering, sub-classifiers are constructed to classify the sub-classes in each cluster. From the experimental results, we found that the label embeddings well reflect the relationships between classification labels, and our ensemble framework can improve the classification performance on a CIFAR 100 dataset.

A Nested Named Entity Recognition Model Robust in Few-shot Learning Environments using Label Information (라벨 정보를 이용한 Few-shot Learning 환경에 강건한 중첩 개체명 인식 모델)

  • Hyunsun Hwang;Changki Lee;Wooyoung Go;Myungchul Kang
    • Annual Conference on Human and Language Technology
    • /
    • 2023.10a
    • /
    • pp.622-626
    • /
    • 2023
  • 중첩 개체명 인식(Nested Named Entity Recognition)은 하나의 개체명 표현 안에 다른 개체명 표현이 들어 있는 중첩 구조의 개체명을 인식하는 작업으로, 중첩 개체명 인식을 위한 학습데이터 구축 작업은 일반 개체명 인식 학습데이터 구축보다 어렵다는 문제가 있다. 본 논문에서는 이러한 문제를 해결하기 위해 Few-shot Learning 환경에 강건한 중첩 개체명 인식 모델을 제안한다. 이를 위해, 기존의 Biaffine 중첩 개체명 인식 모델의 출력 레이어를 라벨 의미 정보를 활용하도록 변경하여 학습데이터가 적은 환경에서 중첩 개체명 인식의 성능을 향상시키도록 하였다. 실험 결과 GENIA 중첩 개체명 인식 데이터의 5-shot, 10-shot, 20-shot 환경에서 기존의 Biaffine 모델보다 평균 10%p이상의 높은 F1-measure 성능을 보였다.

  • PDF

A New Watermarking Method for Video (동영상을 위한 새로운 워터마킹 방법)

  • Kim, Dug-Ryung;Park, Sung-Han
    • Journal of the Korean Institute of Telematics and Electronics S
    • /
    • v.36S no.12
    • /
    • pp.97-106
    • /
    • 1999
  • We propose a new watermarking method to embed a label in a video which is robust against the change of the group of picture. The proposed method embeds labels in the pixel domain, but detects the label in the DCT frequency domain. For embedding a label, the size of watermark based on the human visual system is calculated to keep a quality of videos. A lookup table haying the pixel patterns and the sequences of a sign of DCT coefficients is used for detecting a label in the DCT frequency domain. In this paper, we analyze bit error rates for labels of videos compressed by MPEG2 using the central limit theorem and compare the simulation results with previous methods.

  • PDF

Semantic Feature Analysis for Multi-Label Text Classification on Topics of the Al-Quran Verses

  • Gugun Mediamer;Adiwijaya
    • Journal of Information Processing Systems
    • /
    • v.20 no.1
    • /
    • pp.1-12
    • /
    • 2024
  • Nowadays, Islamic content is widely used in research, including Hadith and the Al-Quran. Both are mostly used in the field of natural language processing, especially in text classification research. One of the difficulties in learning the Al-Quran is ambiguity, while the Al-Quran is used as the main source of Islamic law and the life guidance of a Muslim in the world. This research was proposed to relieve people in learning the Al-Quran. We proposed a word embedding feature-based on Tensor Space Model as feature extraction, which is used to reduce the ambiguity. Based on the experiment results and the analysis, we prove that the proposed method yields the best performance with the Hamming loss 0.10317.

A Novel Classification Model for Efficient Patent Information Research (효율적인 특허정보 조사를 위한 분류 모형)

  • Kim, Youngho;Park, Sangsung;Jang, Dongsik
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.15 no.4
    • /
    • pp.103-110
    • /
    • 2019
  • A patent contains detailed information of the developed technology and is published to the public. Thus, patents can be used to overcome the limitations of traditional technology trend research and prediction techniques. Recently, due to the advantages of patented analytical methodology, IP R&D is carried out worldwide. The patent is big data and has a huge amount, various domains, and structured and unstructured data characteristics. For this reason, there are many difficulties in collecting and researching patent information. Patent research generally writes the Search formula to collect patent documents from DB. The collected patent documents contain some noise patents that are irrelevant to the purpose of analysis, so they are removed. However, eliminating noise patents is a manual task of reading and classifying technology, which is time consuming and expensive. In this study, we propose a model that automatically classifies The Noise patent for efficient patent information research. The proposed method performs Patent Embedding using Word2Vec and generates Noise seed label. In addition, noise patent classification is performed using the Random forest. The experimental data is published and registered with the USPTO among the patents related to Ocean Surveillance & Tracking Network technology. As a result of experimenting with the proposed model, it showed 73% accuracy with the label actually given by experts.

Approximate Top-k Labeled Subgraph Matching Scheme Based on Word Embedding (워드 임베딩 기반 근사 Top-k 레이블 서브그래프 매칭 기법)

  • Choi, Do-Jin;Oh, Young-Ho;Bok, Kyoung-Soo;Yoo, Jae-Soo
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.8
    • /
    • pp.33-43
    • /
    • 2022
  • Labeled graphs are used to represent entities, their relationships, and their structures in real data such as knowledge graphs and protein interactions. With the rapid development of IT and the explosive increase in data, there has been a need for a subgraph matching technology to provide information that the user is interested in. In this paper, we propose an approximate Top-k labeled subgraph matching scheme that considers the semantic similarity of labels and the difference in graph structure. The proposed scheme utilizes a learning model using FastText in order to consider the semantic similarity of a label. In addition, the label similarity graph(LSG) is used for approximate subgraph matching by calculating similarity values between labels in advance. Through the LSG, we can resolve the limitations of the existing schemes that subgraph expansion is possible only if the labels match exactly. It supports structural similarity for a query graph by performing searches up to 2-hop. Based on the similarity value, we provide k subgraph matching results. We conduct various performance evaluations in order to show the superiority of the proposed scheme.

The Fibonacci Edge Labelings on Fibonacci Trees (피보나치트리에서 피보나치 에지 번호매김방법)

  • Kim, Yong-Seok
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.36 no.6
    • /
    • pp.437-450
    • /
    • 2009
  • In this paper, we propose seven edge labeling methods. The methods produce three case of edge labels-sets of Fibonacci numbers {$F_k|k\;{\geq}\;2$}, {$F_{2k}|k\;{\geq}\;1$} and {$F_{3k+2}|k\;{\geq}\;0$}. When a sort of interconnection network, the circulant graph is designed, these edge labels are used for its jump sequence. As a result, the degree is due to the edge labels.

Fine-grained Named Entity Recognition using Hierarchical Label Embedding (계층적 레이블 임베딩을 이용한 세부 분류 개체명 인식)

  • Kim, Hong-Jin;Kim, Hark-Soo
    • Annual Conference on Human and Language Technology
    • /
    • 2021.10a
    • /
    • pp.251-256
    • /
    • 2021
  • 개체명 인식은 정보 추출의 하위 작업으로, 문서에서 개체명에 해당하는 단어를 찾아 알맞은 개체명을 분류하는 자연어처리 기술이다. 질의 응답, 관계 추출 등과 같은 자연어처리 작업에 대한 관심이 높아짐에 따라 세부 분류 개체명 인식에 대한 수요가 증가했다. 그러나 기존 개체명 인식 성능에 비해 세부 분류 개체명 인식의 성능이 낮다. 이러한 성능 차이의 원인은 세부 분류 개체명 데이터가 불균형하기 때문이다. 본 논문에서는 이러한 데이터 불균형 문제를 해결하기 위해 대분류 개체명 정보를 활용하여 세부 분류 개체명 인식을 수행하는 방법과 대분류 개체명 인식의 오류 전파를 완화하기 위한 2단계 학습 방법을 제안한다. 또한 레이블 주의집중 네트워크 기반의 구조에서 레이블의 공통 요소를 공유하여 세부 분류 개체명 인식에 효과적인 레이블 임베딩 구성 방법을 제안한다.

  • PDF