• 제목/요약/키워드: Labeled Data

검색결과 458건 처리시간 0.023초

A Clustering-based Semi-Supervised Learning through Initial Prediction of Unlabeled Data (미분류 데이터의 초기예측을 통한 군집기반의 부분지도 학습방법)

  • Kim, Eung-Ku;Jun, Chi-Hyuck
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • 제33권3호
    • /
    • pp.93-105
    • /
    • 2008
  • Semi-supervised learning uses a small amount of labeled data to predict labels of unlabeled data as well as to improve clustering performance, whereas unsupervised learning analyzes only unlabeled data for clustering purpose. We propose a new clustering-based semi-supervised learning method by reflecting the initial predicted labels of unlabeled data on the objective function. The initial prediction should be done in terms of a discrete probability distribution through a classification method using labeled data. As a result, clusters are formed and labels of unlabeled data are predicted according to the Information of labeled data in the same cluster. We evaluate and compare the performance of the proposed method in terms of classification errors through numerical experiments with blinded labeled data.

Named entity recognition using transfer learning and small human- and meta-pseudo-labeled datasets

  • Kyoungman Bae;Joon-Ho Lim
    • ETRI Journal
    • /
    • 제46권1호
    • /
    • pp.59-70
    • /
    • 2024
  • We introduce a high-performance named entity recognition (NER) model for written and spoken language. To overcome challenges related to labeled data scarcity and domain shifts, we use transfer learning to leverage our previously developed KorBERT as the base model. We also adopt a meta-pseudo-label method using a teacher/student framework with labeled and unlabeled data. Our model presents two modifications. First, the student model is updated with an average loss from both human- and pseudo-labeled data. Second, the influence of noisy pseudo-labeled data is mitigated by considering feedback scores and updating the teacher model only when below a threshold (0.0005). We achieve the target NER performance in the spoken language domain and improve that in the written language domain by proposing a straightforward rollback method that reverts to the best model based on scarce human-labeled data. Further improvement is achieved by adjusting the label vector weights in the named entity dictionary.

Normalization of Microarray Data: Single-labeled and Dual-labeled Arrays

  • Do, Jin Hwan;Choi, Dong-Kug
    • Molecules and Cells
    • /
    • 제22권3호
    • /
    • pp.254-261
    • /
    • 2006
  • DNA microarray is a powerful tool for high-throughput analysis of biological systems. Various computational tools have been created to facilitate the analysis of the large volume of data produced in DNA microarray experiments. Normalization is a critical step for obtaining data that are reliable and usable for subsequent analysis such as identification of differentially expressed genes and clustering. A variety of normalization methods have been proposed over the past few years, but no methods are still perfect. Various assumptions are often taken in the process of normalization. Therefore, the knowledge of underlying assumption and principle of normalization would be helpful for the correct analysis of microarray data. We present a review of normalization techniques from single-labeled platforms such as the Affymetrix GeneChip array to dual-labeled platforms like spotted array focusing on their principles and assumptions.

XML Repository Model based on the Edge-Labeled Graph (Edge-Labeled Graph를 적용한 XML 저장 모델)

  • 김정희;곽호영
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • 제7권5호
    • /
    • pp.993-1001
    • /
    • 2003
  • A RDB Storage Model based on the Edge-Labeled Graph is suggested for store the XML instance in Relational Databases(RDB). The XML instance being stored is represented by Data Graph based on the Edge-Labeled Graph. Data Path Table, Element, Attribute, and Table Index Table values are extracted. Then Database Schema is defined, and the extracted values are stored using the Mapper. In order to support querry, Repository Model offers the translator translating XQL which is used as query language under XPATH, into SQL. In addition, it creates DBtoXML generator restoring the stored XML instance. As a result, storage relationship between the XML instance and proposed model structure can be expressed in terms of Graph-based Path, and it shows the possibility of easy search of random Element and Attribute information.

A Co-training Method based on Classification Using Unlabeled Data (비분류표시 데이타를 이용하는 분류 기반 Co-training 방법)

  • 윤혜성;이상호;박승수;용환승;김주한
    • Journal of KIISE:Software and Applications
    • /
    • 제31권8호
    • /
    • pp.991-998
    • /
    • 2004
  • In many practical teaming problems including bioinformatics area, there is a small amount of labeled data along with a large pool of unlabeled data. Labeled examples are fairly expensive to obtain because they require human efforts. In contrast, unlabeled examples can be inexpensively gathered without an expert. A common method with unlabeled data for data classification and analysis is co-training. This method uses a small set of labeled examples to learn a classifier in two views. Then each classifier is applied to all unlabeled examples, and co-training detects the examples on which each classifier makes the most confident predictions. After some iterations, new classifiers are learned in training data and the number of labeled examples is increased. In this paper, we propose a new co-training strategy using unlabeled data. And we evaluate our method with two classifiers and two experimental data: WebKB and BIND XML data. Our experimentation shows that the proposed co-training technique effectively improves the classification accuracy when the number of labeled examples are very small.

Performance Analysis of MixMatch-Based Semi-Supervised Learning for Defect Detection in Manufacturing Processes (제조 공정 결함 탐지를 위한 MixMatch 기반 준지도학습 성능 분석)

  • Ye-Jun Kim;Ye-Eun Jeong;Yong Soo Kim
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • 제46권4호
    • /
    • pp.312-320
    • /
    • 2023
  • Recently, there has been an increasing attempt to replace defect detection inspections in the manufacturing industry using deep learning techniques. However, obtaining substantial high-quality labeled data to enhance the performance of deep learning models entails economic and temporal constraints. As a solution for this problem, semi-supervised learning, using a limited amount of labeled data, has been gaining traction. This study assesses the effectiveness of semi-supervised learning in the defect detection process of manufacturing using the MixMatch algorithm. The MixMatch algorithm incorporates three dominant paradigms in the semi-supervised field: Consistency regularization, Entropy minimization, and Generic regularization. The performance of semi-supervised learning based on the MixMatch algorithm was compared with that of supervised learning using defect image data from the metal casting process. For the experiments, the ratio of labeled data was adjusted to 5%, 10%, 25%, and 50% of the total data. At a labeled data ratio of 5%, semi-supervised learning achieved a classification accuracy of 90.19%, outperforming supervised learning by approximately 22%p. At a 10% ratio, it surpassed supervised learning by around 8%p, achieving a 92.89% accuracy. These results demonstrate that semi-supervised learning can achieve significant outcomes even with a very limited amount of labeled data, suggesting its invaluable application in real-world research and industrial settings where labeled data is limited.

Improving Accuracy of Instance Segmentation of Teeth

  • Jongjin Park
    • International Journal of Internet, Broadcasting and Communication
    • /
    • 제16권1호
    • /
    • pp.280-286
    • /
    • 2024
  • In this paper, layered UNet with warmup and dropout tricks was used to segment teeth instantly by using data labeled for each individual tooth and increase performance of the result. The layered UNet proposed before showed very good performance in tooth segmentation without distinguishing tooth number. To do instance segmentation of teeth, we labeled teeth CBCT data according to tooth numbering system which is devised by FDI World Dental Federation notation. Colors for labeled teeth are like AI-Hub teeth dataset. Simulation results show that layered UNet does also segment very well for each tooth distinguishing tooth number by color. Layered UNet model using warmup trick was the best with IoU values of 0.80 and 0.77 for training, validation data. To increase the performance of instance segmentation of teeth, we need more labeled data later. The results of this paper can be used to develop medical software that requires tooth recognition, such as orthodontic treatment, wisdom tooth extraction, and implant surgery.

A XML Instance Repository Model based on the Edge-Labeled Graph (Edge-Labeled 그래프 기반의 XML 인스턴스 저장 모델)

  • Kim Jeong-Hee;Kwak Ho-Young
    • Journal of Internet Computing and Services
    • /
    • 제4권6호
    • /
    • pp.33-42
    • /
    • 2003
  • A XML Instance repository model based on the Edge-Labeled Graph is suggested for storing the XML instance in Relational Databases, This repository model represents the XML instance as a data graph based on the Edge-Labeled Graph, extracts the defined value based on the structure of data path, element, attribute, and table index table presented as database schema, and stores these values using the Mapper module, In order to support querry, XML repository model offers the module translating XQL which is a query language under XPATH to SQL, and has DBtoXML generator module restoring the stored XML instance. As a result, it is possible to represent the storage relationship between the XML instances and the proposed repository model in terms of Graph-based Path, and it shows the possibility of easy search of specific element and attribute information.

  • PDF

A Study on the Development of a Postpartum Depression Scale (산후우울 사정을 위한 도구 개발 연구)

  • 배정이
    • Journal of Korean Academy of Nursing
    • /
    • 제27권3호
    • /
    • pp.588-600
    • /
    • 1997
  • Postpartum depression is one of the most serious problems in maternal health because it affects not only the mother but also her family. Postpartum depression disturbs the maternal-infant interaction and attachment. However, most postpartum depression patients ignore this problem and do not seek treatment. Many clinicians and researchers realiza there is a need to develop a postpartum depression scale. Thus, this study has been designed to development of a postpartum depression scale. Data were collected through a survey over a period of three months. Subjects who participated in the study were 167 Korean mothers in their postpartum period. The author used a convenience sampling method. The analysis of the data was done with SPSS PC/sup +/ for descriptive statistics, item analysis and factor analysis. Initially 62 items were generated from the interview data of eight postpartum depression patients and from a literature review. This preliminary scale was analyzed for reliability and validity. The results of this analysis are as follows. 1. Initially 62 items were analyzed through the Index of Content Validity(CVI) and 48 items were selected. 2. Seven factors were extracted through the principal component analysis, and these contributed 61% of the variance in the total score. Finally 46 items in the scale loaded .41∼ .84 on one of seven factors. 3. Each factor was labeled. Factor 1 was labeled 'emotional phenomena-emotional upset' and included 13 items, factor 2 was labeled' cognitive phenomena-self concept disturbance' and included seven items, factor 3 was labeled 'relationship to baby-negative feeling' and included six items, factor 4 was labeled 'relationship to baby- overload' and included eight items, factor 5 was labeled 'negative maternal identity' and included five items, factor 6 was labeled 'biophysiological phenomena-disturbance of physical functioning' and included four items, and factor 7 was labeled' interpersonal relationship phenomena-blamed others' and included three items. 4. Cronbach Coefficient Alpha for internal consistency was .95 for the total 46 items. Finally, the author suggests that this scale could be adequately applied in assessing the postpartum depression of mothers during the postpartum period. The results of this study can contribute to designing an appropriate postpartum depression prevention strategy.

  • PDF

Development of a Steel Plate Surface Defect Detection System Based on Small Data Deep Learning (소량 데이터 딥러닝 기반 강판 표면 결함 검출 시스템 개발)

  • Gaybulayev, Abdulaziz;Lee, Na-Hyeon;Lee, Ki-Hwan;Kim, Tae-Hyong
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • 제17권3호
    • /
    • pp.129-138
    • /
    • 2022
  • Collecting and labeling sufficient training data, which is essential to deep learning-based visual inspection, is difficult for manufacturers to perform because it is very expensive. This paper presents a steel plate surface defect detection system with industrial-grade detection performance by training a small amount of steel plate surface images consisting of labeled and non-labeled data. To overcome the problem of lack of training data, we propose two data augmentation techniques: program-based augmentation, which generates defect images in a geometric way, and generative model-based augmentation, which learns the distribution of labeled data. We also propose a 4-step semi-supervised learning using pseudo labels and consistency training with fixed-size augmentation in order to utilize unlabeled data for training. The proposed technique obtained about 99% defect detection performance for four defect types by using 100 real images including labeled and unlabeled data.