• Title/Summary/Keyword: Linguistic processing

Search Result 171, Processing Time 0.026 seconds

AMR-CNN: Abstract Meaning Representation with Convolution Neural Network for Toxic Content Detection

  • Ermal Elbasani;Jeong-Dong Kim
    • Journal of Web Engineering
    • /
    • v.21 no.3
    • /
    • pp.677-692
    • /
    • 2022
  • Recognizing the offensive, abusive, and profanity of multimedia content on the web has been a challenge to keep the web environment for user's freedom of speech. As profanity filtering function has been developed and applied in text, audio, and video context in platforms such as social media, entertainment, and education, the number of methods to trick the web-based application also has been increased and became a new issue to be solved. Compared to commonly developed toxic content detection systems that use lexicon and keyword-based detection, this work tries to embrace a different approach by the meaning of the sentence. Meaning representation is a way to grasp the meaning of linguistic input. This work proposed a data-driven approach utilizing Abstract meaning Representation to extract the meaning of the online text content into a convolutional neural network to detect level profanity. This work implements the proposed model in two kinds of datasets from the Offensive Language Identification Dataset and other datasets from the Offensive Hate dataset merged with the Twitter Sentiment Analysis dataset. The results indicate that the proposed model performs effectively, and can achieve a satisfactory accuracy in recognizing the level of online text content toxicity.

Multilingual Knowledge Graphs: Challenges and Opportunities

  • Partha Sarathi Mandal;Sukumar Mandal
    • International Journal of Knowledge Content Development & Technology
    • /
    • v.14 no.4
    • /
    • pp.101-111
    • /
    • 2024
  • Multilingual Knowledge Graphs (MKGs) have emerged as a crucial component in various natural language processing tasks, enabling efficient representation and utilization of structured knowledge across multiple languages. One can get data, information, and knowledge from various sectors, like libraries, archives, institutional repositories, etc. Variable quality of metadata, multilingualism, and semantic diversity make it a challenge to create a digital library and multilingual search facility. To accept these challenges, there is a need to design a framework to integrate various structured and unstructured data sources for integration, unification, and sharing databases. These are controlled using linked data and semantic web approaches. In future, multilingual knowledge graph overcomes all the linguistic nuances, technical barriers like semantic interoperability, data harmonization etc and enhance cooperation and collaboration throughout the world. Through a comprehensive analysis of the current state-of-the-art techniques and ongoing research efforts, this paper aims to offer insights into the future directions and potential advancements in the field of Multilingual Knowledge Graphs. This paper deals with a multilingual knowledge graph and how to build up a multilingual knowledge graph. It also focuses on the various challenges and opportunities for designing multilingual knowledge graphs.

Korean Semantic Role Labeling Using Case Frame Dictionary and Subcategorization (격틀 사전과 하위 범주 정보를 이용한 한국어 의미역 결정)

  • Kim, Wan-Su;Ock, Cheol-Young
    • Journal of KIISE
    • /
    • v.43 no.12
    • /
    • pp.1376-1384
    • /
    • 2016
  • Computers require analytic and processing capability for all possibilities of human expression in order to process sentences like human beings. Linguistic information processing thus forms the initial basis. When analyzing a sentence syntactically, it is necessary to divide the sentence into components, find obligatory arguments focusing on predicates, identify the sentence core, and understand semantic relations between the arguments and predicates. In this study, the method applied a case frame dictionary based on The Korean Standard Dictionary of The National Institute of the Korean Language; in addition, we used a CRF Model that constructed subcategorization of predicates as featured in Korean Lexical Semantic Network (UWordMap) for semantic role labeling. Automatically tagged semantic roles based on the CRF model, which established the information of words, predicates, the case-frame dictionary and hypernyms of words as features, were used. This method demonstrated higher performance in comparison with the existing method, with accuracy rate of 83.13% as compared to 81.2%, respectively.

Computational Processing of Korean Dialogue and the Construction of Its Representation Structure Based on Situational Information (상황정보에 기반한 한국어대화의 전산적 처리와 표상구조의 구축)

  • Lee, Dong-Young
    • The KIPS Transactions:PartB
    • /
    • v.9B no.6
    • /
    • pp.817-826
    • /
    • 2002
  • In Korean dialogue honorification phenomenon may occur, an honorific pronoun may be used, and a subject or an object may be completely omitted when it can be recovered based on context. This paper proposes that in order to process Korean dialogue in which such distinct linguistic phenomena occur and to construct its representation structure we mark and use the following information explicitly, not implicitly : information about dialogue participants, information about the speech act of an utterance, information about the relative order of social status for the people involved in dialogue, and information flow among utterances of dialogue. In addition, this paper presents a method of marking and using such situational information and an appropriate representation structure of Korean dialogue. In this paper we set up Korean dialogue representation structure by modifying and extending DRT (Discourse Representation Theory) and SDRT (Segmented Discourse Representation Theory). Futhermore, this paper shows how to process Korean dialogue computationally and construct its representation structure by using Prolog programming language, and then applies such representation structure to spontaneous Korean dialogue to know its validity.

A Development of the Automatic Predicate-Argument Analyzer for Construction of Semantically Tagged Korean Corpus (한국어 의미 표지 부착 말뭉치 구축을 위한 자동 술어-논항 분석기 개발)

  • Cho, Jung-Hyun;Jung, Hyun-Ki;Kim, Yu-Seop
    • The KIPS Transactions:PartB
    • /
    • v.19B no.1
    • /
    • pp.43-52
    • /
    • 2012
  • Semantic role labeling is the research area analyzing the semantic relationship between elements in a sentence and it is considered as one of the most important semantic analysis research areas in natural language processing, such as word sense disambiguation. However, due to the lack of the relative linguistic resources, Korean semantic role labeling research has not been sufficiently developed. We, in this paper, propose an automatic predicate-argument analyzer to begin constructing the Korean PropBank which has been widely utilized in the semantic role labeling. The analyzer has mainly two components: the semantic lexical dictionary and the automatic predicate-argument extractor. The dictionary has the case frame information of verbs and the extractor is a module to decide the semantic class of the argument for a specific predicate existing in the syntactically annotated corpus. The analyzer developed in this research will help the construction of Korean PropBank and will finally play a big role in Korean semantic role labeling.

A Study of Morphophonemic Processes of Korean using Neural Networks (인공신경망을 이용한 한국어 형태음운현상 연구)

  • Lee, Chan-Do
    • The Transactions of the Korea Information Processing Society
    • /
    • v.2 no.2
    • /
    • pp.215-228
    • /
    • 1995
  • Despite their importance in language, there have been relatively few computational studies in understanding words. This paper describes how neural networks can learn to perceive and produce words. Most traditional linguistic theories presuppose abstract underlying representations (UR) and a set of explicit rules to obtain the surface realization. There are, however, a number of questions that can be raised regarding this approach: (1) assumption of URs, (2) formation of rules, and (3) interaction of rules. In this paper, it is hypothesized that rules would emerge as the generalizations the network abstracts in the process of learning to associate forms with meanings of the words. Employing a simple recurrent network, a series of simulations on different types of morphophonemic processes was run. The results of the simulations show that this network is capable of learning to perceive whether words are in basic from or in inflected form, given only forms, and to produce words in the right form, given arbitrary meanings, this eliminating the need for presupposing abstract URs and rules.

  • PDF

Efficient Inference of Image Objects using Semantic Segmentation (시멘틱 세그멘테이션을 활용한 이미지 오브젝트의 효율적인 영역 추론)

  • Lim, Heonyeong;Lee, Yurim;Jee, Minkyu;Go, Myunghyun;Kim, Hakdong;Kim, Wonil
    • Journal of Broadcast Engineering
    • /
    • v.24 no.1
    • /
    • pp.67-76
    • /
    • 2019
  • In this paper, we propose an efficient object classification method based on semantic segmentation for multi-labeled image data. In addition to various pixel unit information and processing techniques such as color information, contour, contrast, and saturation included in image data, a detailed region in which each object is located is extracted as a meaningful unit and the experiment is conducted to reflect the result in the inference. We use a neural network that has been proven to perform well in image classification to understand which object is located where image data containing various class objects are located. Based on these researches, we aim to provide artificial intelligence services that can classify real-time detailed areas of complex images containing various objects in the future.

Predicate Recognition Method using BiLSTM Model and Morpheme Features (BiLSTM 모델과 형태소 자질을 이용한 서술어 인식 방법)

  • Nam, Chung-Hyeon;Jang, Kyung-Sik
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.26 no.1
    • /
    • pp.24-29
    • /
    • 2022
  • Semantic role labeling task used in various natural language processing fields, such as information extraction and question answering systems, is the task of identifying the arugments for a given sentence and predicate. Predicate used as semantic role labeling input are extracted using lexical analysis results such as POS-tagging, but the problem is that predicate can't extract all linguistic patterns because predicate in korean language has various patterns, depending on the meaning of sentence. In this paper, we propose a korean predicate recognition method using neural network model with pre-trained embedding models and lexical features. The experiments compare the performance on the hyper parameters of models and with or without the use of embedding models and lexical features. As a result, we confirm that the performance of the proposed neural network model was 92.63%.

Sentence Filtering Dataset Construction Method about Web Corpus (웹 말뭉치에 대한 문장 필터링 데이터 셋 구축 방법)

  • Nam, Chung-Hyeon;Jang, Kyung-Sik
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.11
    • /
    • pp.1505-1511
    • /
    • 2021
  • Pretrained models with high performance in various tasks within natural language processing have the advantage of learning the linguistic patterns of sentences using large corpus during the training, allowing each token in the input sentence to be represented with appropriate feature vectors. One of the methods of constructing a corpus required for a pre-trained model training is a collection method using web crawler. However, sentences that exist on web may contain unnecessary words in some or all of the sentences because they have various patterns. In this paper, we propose a dataset construction method for filtering sentences containing unnecessary words using neural network models for corpus collected from the web. As a result, we construct a dataset containing a total of 2,330 sentences. We also evaluated the performance of neural network models on the constructed dataset, and the BERT model showed the highest performance with an accuracy of 93.75%.

A Comparative Study on the Pronunciations of Korean and Vietnamese on Korean Syllable Final Double Consonants (베트남인 한국어 학습자와 한국인의 한국어 겹받침 발음 비교 연구)

  • Jang, Kyungnam;You, Kwang-Bock
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.6
    • /
    • pp.637-646
    • /
    • 2022
  • In this paper the comparative study on the pronunciation of Vietnamese learners and Koreans for the Korean syllable final double consonants was performed. For many errors and the suggested teaching methods related to the pronunciation of the Korean syllable final double consonants that were investigated and analyzed through linguistic research the results of this study by using the analysis tools of speech signal processing were confirmed. Thus, we suggest the new educational method in this paper. Using SVM, which is widely used in machine learning of artificial intelligence the pronunciation of Vietnamese learners and that of Koreans were compared. Being able to obtain the decision hyperplane of the SVM means that Vietnamese learners' pronunciation of the Korean syllable final double consonants is quite different from that of Koreans. Otherwise their pronunciation are pretty similar each other. The new teaching method presented in this paper is not only composed of writing and listening but is included things such as the speech signal waveform in the time domain and its corresponding energy that can be visualized to the learners.