• Title/Summary/Keyword: Part of speech

Search Result 439, Processing Time 0.027 seconds

Language- Independent Sentence Boundary Detection with Automatic Feature Selection

  • Lee, Do-Gil
    • Journal of the Korean Data and Information Science Society
    • /
    • v.19 no.4
    • /
    • pp.1297-1304
    • /
    • 2008
  • This paper proposes a machine learning approach for language-independent sentence boundary detection. The proposed method requires no heuristic rules and language-specific features, such as part-of-speech information, a list of abbreviations or proper names. With only the language-independent features, we perform experiments on not only an inflectional language but also an agglutinative language, having fairly different characteristics (in this paper, English and Korean, respectively). In addition, we obtain good performances in both languages. We have also experimented with the methods under a wide range of experimental conditions, especially for the selection of useful features.

  • PDF

A Prior Model of Structural SVMs for Domain Adaptation

  • Lee, Chang-Ki;Jang, Myung-Gil
    • ETRI Journal
    • /
    • v.33 no.5
    • /
    • pp.712-719
    • /
    • 2011
  • In this paper, we study the problem of domain adaptation for structural support vector machines (SVMs). We consider a number of domain adaptation approaches for structural SVMs and evaluate them on named entity recognition, part-of-speech tagging, and sentiment classification problems. Finally, we show that a prior model for structural SVMs outperforms other domain adaptation approaches in most cases. Moreover, the training time for this prior model is reduced compared to other domain adaptation methods with improvements in performance.

Text Categorization Based on the Maximum Entropy Principle (최대 엔트로피 기반 문서 분류기의 학습)

  • 장정호;장병탁;김영택
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 1999.10b
    • /
    • pp.57-59
    • /
    • 1999
  • 본 논문에서는 최대 엔트로피 원리에 기반한 문서 분류기의 학습을 제안한다. 최대 엔트로피 기법은 자연언어 처리에서 언어 모델링(Language Modeling), 품사 태깅 (Part-of-Speech Tagging) 등에 널리 사용되는 방법중의 하나이다. 최대 엔트로피 모델의 효율성을 위해서는 자질 선정이 중요한데, 본 논문에서는 자질 집합의 선택을 위한 기준으로 chi-square test, log-likelihood ratio, information gain, mutual information 등의 방법을 이용하여 실험하고, 전체 후보 자질에 대한 실험 결과와 비교해 보았다. 데이터 집합으로는 Reuters-21578을 사용하였으며, 각 클래스에 대한 이진 분류 실험을 수행하였다.

  • PDF

Resolving Part-of-Speech Tagging Ambiguities by a Maximum Entropy Boosting Model (최대 엔트로피 부스팅 모델을 이용한 품사 모호성 해소)

  • 박성배;장병탁
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2003.04c
    • /
    • pp.522-524
    • /
    • 2003
  • 품사 결정 문제는 자연언어처리의 가장 기본적인 문제들 중 하나이며, 기계학습의 관점에서 보면 분류 문제(classification problem)로 쉽게 표현된다. 본 논문에서는 품사 결정의 모호성을 해소하기 위해서 최대 엔트로피 부스팅 모델(maximum entropy boosting model)을 이 문제에 적응하였다. 그리고, 품사 결정에서 중요한 요소 중의 하나인 미지어 처리를 위해서 특별히 설계된 일차 자질을 고려하였다. 최대 엔트로피 부스팅 모델의 장점은 쉬운 모델링인데, 실제로 품사 결정을 위한 일차 자질만 작성하는 노려만 들이고도 96.78%의 정확도를 보여 지금까지 알려진 최고의 성능과 거의 비슷한 결과를 보였다.

  • PDF

Korean Part-of-Speech Tagging using Constrained-Rule and Main POS Information among Words (어절간 주품사 정보와 제약 규칙을 이용한 한국어 품사 태깅 시스템)

  • Kang, Yu-Hwan;Seo, Young-Hoon
    • Annual Conference on Human and Language Technology
    • /
    • 1999.10e
    • /
    • pp.433-437
    • /
    • 1999
  • 본 논문에서는 품사 태깅을 위한 방법으로 어절간 품사 패턴 정보를 이용하는 방법을 제안한다. 품사 태깅을 위하여 여러 어절들 간의 품사 패턴 정보를 통계 정보로 구축하고 품사 태깅시에 품사 패턴 정보를 이용하여 품사 태깅을 수행한다. 이때 품사 패턴 적용시 몇가지 제약 규칙을 둠으로써 품사 태깅의 정확률을 높이는 방법을 연구하였다.

  • PDF

Extraction of the Latent Index Terms Using the Word Frequency and Part of Speech in Automatic Indexing (자동색인에서 단어의 품사와 빈도를 이용한 색인후보어 발췌)

  • 이태영;남궁황
    • Proceedings of the Korean Society for Information Management Conference
    • /
    • 2001.08a
    • /
    • pp.181-184
    • /
    • 2001
  • 본 논문에서는 적합한 색인어를 자동으로 추출해 내기 위해 잘 알려진 통계적 기법과 구문분석적 기법을 혼용하였다. 적용결과를 검색효율로 나타내지 않고 각 방법에 따라 추출된 단어들을 실증적으로 보여주어 성능에 대한 판단을 유도하였다. 빈도나 품사가 단독으로 사용된 것보다 동시에 적용된 것이 보다 좋은 결과를 가져왔다.

  • PDF

Evolution and current status of microsurgical tongue reconstruction, part II

  • Choi, Jong-Woo;Alshomer, Feras;Kim, Young-Chul
    • Archives of Craniofacial Surgery
    • /
    • v.23 no.5
    • /
    • pp.193-204
    • /
    • 2022
  • Tongue reconstruction remains a major aspect of head and neck reconstructive procedures. Surgeons planning tongue reconstruction should consider several factors to optimize the overall outcomes. Specifically, various technical aspects related to tongue reconstruction have been found to affect the outcomes. Multidisciplinary teams dedicated to oncologic, reconstructive, and rehabilitative approaches play an essential role in the reconstructive process. Moreover, operative planning addressing certain patient-related and defect-related factors is crucial for optimizing functional speech and swallowing, as well as quality of life outcomes. Furthermore, tongue reconstruction is a delicate process, in which overall functional outcomes result from proper flap selection and shaping, recipient vessel preparation and anastomosis, surgical approaches to flap insetting, and postoperative management. The second part of this review summarizes these factors in relation to tongue reconstruction.

A Semi-supervised Learning of HMM to Build a POS Tagger for a Low Resourced Language

  • Pattnaik, Sagarika;Nayak, Ajit Kumar;Patnaik, Srikanta
    • Journal of information and communication convergence engineering
    • /
    • v.18 no.4
    • /
    • pp.207-215
    • /
    • 2020
  • Part of speech (POS) tagging is an indispensable part of major NLP models. Its progress can be perceived on number of languages around the globe especially with respect to European languages. But considering Indian Languages, it has not got a major breakthrough due lack of supporting tools and resources. Particularly for Odia language it has not marked its dominancy yet. With a motive to make the language Odia fit into different NLP operations, this paper makes an attempt to develop a POS tagger for the said language on a HMM (Hidden Markov Model) platform. The tagger judiciously considers bigram HMM with dynamic Viterbi algorithm to give an output annotated text with maximum accuracy. The model is experimented on a corpus belonging to tourism domain accounting to a size of approximately 0.2 million tokens. With the proportion of training and testing as 3:1, the proposed model exhibits satisfactory result irrespective of limited training size.

The cerebral representation related to lexical ambiguity and idiomatic ambiguity (어휘적 중의성 및 관용적 중의성을 처리하는 대뇌 영역)

  • Yu Gisoon;Kang Hongmo;Jo Kyungduk;Kang Myungyoon;Nam Kichun
    • Proceedings of the KSPS conference
    • /
    • 2003.10a
    • /
    • pp.79-82
    • /
    • 2003
  • The purpose of this study is to examine the regions of the cerebrum that handles the lexical and idiomatic ambiguity. The stimuli sets consist of two parts, and each part has 20 sets of sentences. For each part, 10 sets are experimental conditions and the other 10 sets are control conditions. Each set has two sentences, the 'context' and 'target' sentences, and a sentence-verification question for guaranteeing patients' concentration to the task. The results based on 15 patients showed that significant activation is present in the right frontal lobe of the cerebral cortex for both kinds of ambiguity. It means that right hemisphere participates in the resolution of ambiguity, and there are no regions specified for lexical ambiguity or idiomatic ambiguity alone.

  • PDF

Topic Analysis of the National Petition Site and Prediction of Answerable Petitions Based on Deep Learning (국민청원 주제 분석 및 딥러닝 기반 답변 가능 청원 예측)

  • Woo, Yun Hui;Kim, Hyon Hee
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.9 no.2
    • /
    • pp.45-52
    • /
    • 2020
  • Since the opening of the national petition site, it has attracted much attention. In this paper, we perform topic analysis of the national petition site and propose a prediction model for answerable petitions based on deep learning. First, 1,500 petitions are collected, topics are extracted based on the petitions' contents. Main subjects are defined using K-means clustering algorithm, and detailed subjects are defined using topic modeling of petitions belonging to the main subjects. Also, long short-term memory (LSTM) is used for prediction of answerable petitions. Not only title and contents but also categories, length of text, and ratio of part of speech such as noun, adjective, adverb, verb are also used for the proposed model. Our experimental results show that the type 2 model using other features such as ratio of part of speech, length of text, and categories outperforms the type 1 model without other features.