• Title/Summary/Keyword: BERT

Search Result 395, Processing Time 0.025 seconds

Cross-Layer Reduction of Wireless Network Card Idle Time to Optimize Energy Consumption of Pull Thin Client Protocols

  • Simoens, Pieter;Ali, Farhan Azmat;Vankeirsbilck, Bert;Deboosere, Lien;Turck, Filip De;Dhoedt, Bart;Demeester, Piet;Torrea-Duran, Rodolfo;Perre, Liesbet Van der;Dejonghe, Antoine
    • Journal of Communications and Networks
    • /
    • v.14 no.1
    • /
    • pp.75-90
    • /
    • 2012
  • Thin client computing trades local processing for network bandwidth consumption by offloading application logic to remote servers. User input and display updates are exchanged between client and server through a thin client protocol. On wireless devices, the thin client protocol traffic can lead to a significantly higher power consumption of the radio interface. In this article, a cross-layer framework is presented that transitions the wireless network interface card (WNIC) to the energy-conserving sleep mode when no traffic from the server is expected. The approach is validated for different wireless channel conditions, such as path loss and available bandwidth, as well as for different network roundtrip time values. Using this cross-layer algorithm for sample scenario with a remote text editor, and through experiments based on actual user traces, a reduction of the WNIC energy consumption of up to 36.82% is obtained, without degrading the application's reactivity.

Implemeention and performance measurement of a novel in-service supervisory system for WDM transmission link (파장분할다중화방식 전송로의 In-service 감시를 위한 새로운 감시시스템의 구현 및 성능평가)

  • 김필한;윤호성;박남규;서재은;정기태;유기원;이규행
    • Korean Journal of Optics and Photonics
    • /
    • v.12 no.2
    • /
    • pp.129-134
    • /
    • 2001
  • Novel supervisory system for WDM transmission link using conventional optical time domain reflectometry was presented. By modifying the structure of erbuim doped fiber amplifier to support bi-directional transmission at arDR pulse wavelength and launching the optical pulse into transmission link in the opposite direction of data signal propagation to avoid the distortion by cross-gain modulation, it is possible to monitor the WDM link in service. To prove the validity of proposed scheme, the supervision result of 2.5 Gbps $\times$ 8 channel WDM 320 km transmission system in service by arDR was presented. And power penalty due to monitoring was measured as smaller than 0.3 dB. .3 dB.

  • PDF

Development of Korean dataset for joint intent classification and slot filling (발화 의도 예측 및 슬롯 채우기 복합 처리를 위한 한국어 데이터셋 개발)

  • Han, Seunggyu;Lim, Heuiseok
    • Journal of the Korea Convergence Society
    • /
    • v.12 no.1
    • /
    • pp.57-63
    • /
    • 2021
  • Spoken language understanding, which aims to understand utterance as naturally as human would, are mostly focused on English language. In this paper, we construct a Korean language dataset for spoken language understanding, which is based on a conversational corpus between reservation system and its user. The domain of conversation is limited to restaurant reservation. There are 7 types of slot tags and 5 types of intent tags in 6857 sentences. When a model proposed in English-based research is trained with our dataset, intent classification accuracy decreased a little, while slot filling F1 score decreased significantly.

Attention Capsule Network for Aspect-Level Sentiment Classification

  • Deng, Yu;Lei, Hang;Li, Xiaoyu;Lin, Yiou;Cheng, Wangchi;Yang, Shan
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.4
    • /
    • pp.1275-1292
    • /
    • 2021
  • As a fine-grained classification problem, aspect-level sentiment classification predicts the sentiment polarity for different aspects in context. To address this issue, researchers have widely used attention mechanisms to abstract the relationship between context and aspects. Still, it is difficult to effectively obtain a more profound semantic representation, and the strong correlation between local context features and the aspect-based sentiment is rarely considered. In this paper, a hybrid attention capsule network for aspect-level sentiment classification (ABASCap) was proposed. In this model, the multi-head self-attention was improved, and a context mask mechanism based on adjustable context window was proposed, so as to effectively obtain the internal association between aspects and context. Moreover, the dynamic routing algorithm and activation function in capsule network were optimized to meet the task requirements. Finally, sufficient experiments were conducted on three benchmark datasets in different domains. Compared with other baseline models, ABASCap achieved better classification results, and outperformed the state-of-the-art methods in this task after incorporating pre-training BERT.

A Protein-Protein Interaction Extraction Approach Based on Large Pre-trained Language Model and Adversarial Training

  • Tang, Zhan;Guo, Xuchao;Bai, Zhao;Diao, Lei;Lu, Shuhan;Li, Lin
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.3
    • /
    • pp.771-791
    • /
    • 2022
  • Protein-protein interaction (PPI) extraction from original text is important for revealing the molecular mechanism of biological processes. With the rapid growth of biomedical literature, manually extracting PPI has become more time-consuming and laborious. Therefore, the automatic PPI extraction from the raw literature through natural language processing technology has attracted the attention of the majority of researchers. We propose a PPI extraction model based on the large pre-trained language model and adversarial training. It enhances the learning of semantic and syntactic features using BioBERT pre-trained weights, which are built on large-scale domain corpora, and adversarial perturbations are applied to the embedding layer to improve the robustness of the model. Experimental results showed that the proposed model achieved the highest F1 scores (83.93% and 90.31%) on two corpora with large sample sizes, namely, AIMed and BioInfer, respectively, compared with the previous method. It also achieved comparable performance on three corpora with small sample sizes, namely, HPRD50, IEPA, and LLL.

Modern Methods of Text Analysis as an Effective Way to Combat Plagiarism

  • Myronenko, Serhii;Myronenko, Yelyzaveta
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.8
    • /
    • pp.242-248
    • /
    • 2022
  • The article presents the analysis of modern methods of automatic comparison of original and unoriginal text to detect textual plagiarism. The study covers two types of plagiarism - literal, when plagiarists directly make exact copying of the text without changing anything, and intelligent, using more sophisticated techniques, which are harder to detect due to the text manipulation, like words and signs replacement. Standard techniques related to extrinsic detection are string-based, vector space and semantic-based. The first, most common and most successful target models for detecting literal plagiarism - N-gram and Vector Space are analyzed, and their advantages and disadvantages are evaluated. The most effective target models that allow detecting intelligent plagiarism, particularly identifying paraphrases by measuring the semantic similarity of short components of the text, are investigated. Models using neural network architecture and based on natural language sentence matching approaches such as Densely Interactive Inference Network (DIIN), Bilateral Multi-Perspective Matching (BiMPM) and Bidirectional Encoder Representations from Transformers (BERT) and its family of models are considered. The progress in improving plagiarism detection systems, techniques and related models is summarized. Relevant and urgent problems that remain unresolved in detecting intelligent plagiarism - effective recognition of unoriginal ideas and qualitatively paraphrased text - are outlined.

Analysis of International Research Trends on Metaverse

  • Mina, Shim
    • International Journal of Advanced Culture Technology
    • /
    • v.10 no.4
    • /
    • pp.453-459
    • /
    • 2022
  • This study attempted to explore the realization and research direction of a successful metaverse environment in the future by analyzing international research trends of the metaverse using topic modeling. A total of 208 papers among WoS and ScienceDirect papers using metaverse as keywords were selected, and quantitative frequency analysis and topic modeling were performed. As a result, it was confirmed that research has rapidly increased after 2022. The main keywords of the research topics were 'second', 'life', 'learning', 'reality', 'metaverse', 'virtual', 'blockchain', 'nft', 'medical', 'avatar', etc. The topic keywords 'Second life & Education' and 'Virtual Reality & Medical' accounted for a large proportion of 57%, followed by 'Blockchain & Cryptocurrency', 'Avatar & Interaction', and 'Sensing and Device'. As a result of semantic analysis, current metaverse research is focused on application and utilization, and research on underlying technologies and devices is also active. Therefore, it is necessary to identify the commonalities and differences between domestic and foreign studies, and to study the application method considering the domestic environment. In addition, new jurisprudence research is more necessary along with predicting new problems. It is expected that the results of study will provide the right research direction for domestic researchers in the era of digital transformation and contribute to the realization of a digital society.

News Recommendation Exploiting Document Summarization based on Deep Learning (딥러닝 기반의 문서요약기법을 활용한 뉴스 추천)

  • Heu, Jee-Uk
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.22 no.4
    • /
    • pp.23-28
    • /
    • 2022
  • Recently smart device(such as smart phone and tablet PC) become a role as an information gateway, using of the web news by multiple users from the web portal has been more important things. However, the quantity of creating web news on the web makes hard to catch the information which the user wants and confuse the users cause of the similar and repeated contents. In this paper, we propose the news recommend system using the document summarization based on KoBART which gives the selected news to users from the candidate news on the news portal. As a result, our proposed system shows higher performance and recommending the news efficiently by pre-training and fine-tuning the KoBART using collected news data.

A Method of Classification of Overseas Direct Purchase Product Groups Based on Transfer Learning (언어모델 전이학습 기반 해외 직접 구매 상품군 분류)

  • Kyo-Joong Oh;Ho-Jin Choi;Wonseok Cha;Ilgu Kim;Chankyun Woo
    • Annual Conference on Human and Language Technology
    • /
    • 2022.10a
    • /
    • pp.571-575
    • /
    • 2022
  • 본 논문에서는 통계청에서 매월 작성되는 온라인쇼핑동향조사를 위해, 언어모델 전이학습 기반 분류모델 학습 방법론을 이용하여, 관세청 제공 전자상거래 수입 목록통관 자료를 처리하기 위해서 해외 직접 구매 상품군 분류 모델을 구축한다. 최근에 텍스트 분류 태스크에서 많이 이용되는 BERT 기반의 언어모델을 이용하며 기존의 색인어 정보 분석 과정이나 사례사전 구축 등의 중간 단계 없이 해외 직접 판매 및 구매 상품군을 94%라는 높은 예측 정확도로 분류가 가능해짐을 알 수 있다.

  • PDF

Parameter-Efficient Prompting for Few-Shot Learning (Prompting 기반 매개변수 효율적인 Few-Shot 학습 연구)

  • Eunhwan Park;Sung-Min Lee;Daeryong Seo;Donghyeon Jeon;Inho Kang;Seung-Hoon Na
    • Annual Conference on Human and Language Technology
    • /
    • 2022.10a
    • /
    • pp.343-347
    • /
    • 2022
  • 최근 자연어처리 분야에서는 BERT, RoBERTa, 그리고 BART와 같은 사전 학습된 언어 모델 (Pre-trained Language Models, PLM) 기반 미세 조정 학습을 통하여 여러 하위 과업에서 좋은 성능을 거두고 있다. 이는 사전 학습된 언어 모델 및 데이터 집합의 크기, 그리고 모델 구성의 중요성을 보여주며 대규모 사전 학습된 언어 모델이 각광받는 계기가 되었다. 하지만, 거대한 모델의 크기로 인하여 실제 산업에서 쉽게 쓰이기 힘들다는 단점이 명백히 존재함에 따라 최근 매개변수 효율적인 미세 조정 및 Few-Shot 학습 연구가 많은 주목을 받고 있다. 본 논문은 Prompt tuning, Prefix tuning와 프롬프트 기반 미세 조정 (Prompt-based fine-tuning)을 결합한 Few-Shot 학습 연구를 제안한다. 제안한 방법은 미세 조정 ←→ 사전 학습 간의 지식 격차를 줄일 뿐만 아니라 기존의 일반적인 미세 조정 기반 Few-Shot 학습 성능보다 크게 향상됨을 보인다.

  • PDF