• Title/Summary/Keyword: Hierarchical Attention Networks

Search Result 17, Processing Time 0.025 seconds

Multi-task learning with contextual hierarchical attention for Korean coreference resolution

  • Cheoneum Park
    • ETRI Journal
    • /
    • v.45 no.1
    • /
    • pp.93-104
    • /
    • 2023
  • Coreference resolution is a task in discourse analysis that links several headwords used in any document object. We suggest pointer networks-based coreference resolution for Korean using multi-task learning (MTL) with an attention mechanism for a hierarchical structure. As Korean is a head-final language, the head can easily be found. Our model learns the distribution by referring to the same entity position and utilizes a pointer network to conduct coreference resolution depending on the input headword. As the input is a document, the input sequence is very long. Thus, the core idea is to learn the word- and sentence-level distributions in parallel with MTL, while using a shared representation to address the long sequence problem. The suggested technique is used to generate word representations for Korean based on contextual information using pre-trained language models for Korean. In the same experimental conditions, our model performed roughly 1.8% better on CoNLL F1 than previous research without hierarchical structure.

Phrase-Chunk Level Hierarchical Attention Networks for Arabic Sentiment Analysis

  • Abdelmawgoud M. Meabed;Sherif Mahdy Abdou;Mervat Hassan Gheith
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.9
    • /
    • pp.120-128
    • /
    • 2023
  • In this work, we have presented ATSA, a hierarchical attention deep learning model for Arabic sentiment analysis. ATSA was proposed by addressing several challenges and limitations that arise when applying the classical models to perform opinion mining in Arabic. Arabic-specific challenges including the morphological complexity and language sparsity were addressed by modeling semantic composition at the Arabic morphological analysis after performing tokenization. ATSA proposed to perform phrase-chunks sentiment embedding to provide a broader set of features that cover syntactic, semantic, and sentiment information. We used phrase structure parser to generate syntactic parse trees that are used as a reference for ATSA. This allowed modeling semantic and sentiment composition following the natural order in which words and phrase-chunks are combined in a sentence. The proposed model was evaluated on three Arabic corpora that correspond to different genres (newswire, online comments, and tweets) and different writing styles (MSA and dialectal Arabic). Experiments showed that each of the proposed contributions in ATSA was able to achieve significant improvement. The combination of all contributions, which makes up for the complete ATSA model, was able to improve the classification accuracy by 3% and 2% on Tweets and Hotel reviews datasets, respectively, compared to the existing models.

On the Design of Delay based Admission Control in Hierarchical Networks

  • Shin, Seungjae;Kim, Namgi;Lee, Byoung-Dai;Choi, Yoon-Ho;Yoon, Hyunsoo
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.8 no.3
    • /
    • pp.997-1010
    • /
    • 2014
  • Today, as the hierarchical cellular system is getting more attention than before, some recent studies introduce delay based admission control (AC) scheme which delays the admission to the macro-embedded small cell for a relatively short time to prevent unnecessary handover caused by the short-term visitors of the small cell area. In such delay based ACs, when we use improper delay parameter, the system frequently makes incorrect handover decisions such as where unnecessary handover is allowed due to too short delaying, or where necessary handover is denied due to too long delaying. In order to avoid these undesirable situations as much as possible, we develop a new delay parameter decision method based on probabilistic cell residence time approximations. By the extensive numerical and analytical evaluations, we determine the proper delay parameter which prevents the incorrect handover decision as much as possible. We expect our delay parameter decision method can be useful system administration tips in hierarchical cellular system where delay based AC is adopted.

System Capacity and Coverage Analysis of Hierarchical Femtocell Networks (펨토셀 기반 계층셀 구조 시스템 용량 및 서비스 반경 분석)

  • O, Nam-Geol;Kim, Hoon
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.34 no.6A
    • /
    • pp.476-483
    • /
    • 2009
  • Recently much attention has been devoted to femtocell's potential to improve indoor cellular coverage and high speed wireless communications. Femtocell based commercial services have been already launched in some countries and standardization activities are actively on-going, there has been concern however over potential issues of interference between femtocells and the micro/macro networks. With universal frequency reuse, the ensuing cross-tier interference causes unacceptable data rate and outage probability, so an analysis of effect of interference in femtocell embedded networks would be necessary for a stable system design. This paper investigates the effect of interference on system performances of femtocell embedded hierarchical cell structure (HCS) networks considering the characteristics of propagation environments. Various channel parameters are specially considered for indoor environments where most of femtocells are deployed to investigate the effect of interference of femtocell embedded RCS networks. System capacity and coverage are provided with variant distance between macrocell and femtocell, location of the user in femtocell coverage, and characteristic of building structures.

An Architecture for Key Management in Hierarchical Mobile Ad-hoc Networks

  • Rhee, Kyung-Hyune;Park, Young-Ho;Gene Tsudik
    • Journal of Communications and Networks
    • /
    • v.6 no.2
    • /
    • pp.156-162
    • /
    • 2004
  • In recent years, mobile ad-hoc networks have received a great deal of attention in both academia and industry to provide anytime-anywhere networking services. As wireless networks are rapidly deployed, the security of wireless environment will be mandatory. In this paper, we describe a group key management architecture and key agreement protocols for secure communication in mobile ad-hoc wireless networks (MANETs) overseen by unmanned aerial vehicles (UAVs). We use implicitly certified public keys method, which alleviates the certificate overhead and improves computational efficiency. The architecture uses a two-layered key management approach where the group of nodes is divided into: 1) Cell groups consisting of ground nodes and 2) control groups consisting of cell group managers. The chief benefit of this approach is that the effects of a membership change are restricted to the single cell group.

Topic Modeling with Deep Learning-based Sentiment Filters (감정 딥러닝 필터를 활용한 토픽 모델링 방법론)

  • Choi, Byeong-Seol;Kim, Namgyu
    • The Journal of Information Systems
    • /
    • v.28 no.4
    • /
    • pp.271-291
    • /
    • 2019
  • Purpose The purpose of this study is to propose a methodology to derive positive keywords and negative keywords through deep learning to classify reviews into positive reviews and negative ones, and then refine the results of topic modeling using these keywords. Design/methodology/approach In this study, we extracted topic keywords by performing LDA-based topic modeling. At the same time, we performed attention-based deep learning to identify positive and negative keywords. Finally, we refined the topic keywords using these keywords as filters. Findings We collected and analyzed about 6,000 English reviews of Gyeongbokgung, a representative tourist attraction in Korea, from Tripadvisor, a representative travel site. Experimental results show that the proposed methodology properly identifies positive and negative keywords describing major topics.

Network Mobility Support Using Router-based Binding Update Scheme in Hierarchical Mobile IP (Hierarchical Mobile IP에서 라우터기반 바인딩 업데이트 방법을 이용한 네트워크 이동성 지원)

  • Kim, Ju-Hyun;Lee, Kyung-Geun
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.34 no.7B
    • /
    • pp.668-676
    • /
    • 2009
  • Hierarchical Mobile 1Pv6(HM1Pv6) have been proposed by IETF(Internet Engineering Task Force) beginning from Mobile 1Pv6 for mobility support in IP networks, however these researches relying on client based mobility support have not been drawn attention due to excessive consumption of wireless resources and long handover delay. In this paper, we propose the Router-based Binding Update(RBU) scheme to solve problems in existing client-based mobility support schemes. The router registers location information of mobile nodes through Neighbor Discovery protocol without additional codes for the RBU scheme to a terminal operated by existing HMlPv6. By using this the RBU scheme is designed so that it can support partial nupport based mobility and reduce handover latency rather than using HMIPv6. It is analysed and compared with existing HMIPv6 to verify efficiency of the RBU scheme. As a result, the RBU scheme has outperformed existing HMIPv6 by 15% in terms of macro handover delay especially when long delay on wireless links exists.

Design of Service Signaling Structure based on MMT for Terrestrial UHD Broadcasting Systems in Heterogeneous Network (이기종망 환경에서의 지상파 UHD 방송을 위한 MMT 기반 서비스 시그널링 구조 설계)

  • Seo, Minjae;Paik, Jong-Ho
    • Journal of Satellite, Information and Communications
    • /
    • v.10 no.2
    • /
    • pp.54-59
    • /
    • 2015
  • UHD(Ultra High Definition) Broadcasting is the focus of attention recent days. However, current broadcasting system faces with difficulty of bandwidth, encoding and so on. To solve this problem, MMT was suggested as the one of solutions. MMT is designed to be based on IP network and it has characteristics that can deliver multimedia through many networks at the same time. UHD Media service can be available with the current broadcasting system that divides media data into hierarchical data based on MMT. To provide this service, information about heterogeneous network should be delivered and signalling should be given to perceive it. For UHD media data service, the information about data from heterogeneous networks should be transported for providing presentation information and service for the receiver models. The present MMT signalling has not much information about heterogeneous services with hierarchical media data. In this paper, we suggest the design of service signaling structure based on MMT for UHD broadcasting systems.

Hierarchical attention based CNN-RNN networks for The Korean Speech-Act Analysis (계층 구조 어텐션 매커니즘에 기반한 CNN-RNN을 이용한 한국어 화행 분석 시스템)

  • Seo, Minyeong;Hong, Taesuk;Kim, Juae;Ko, Youngjoong;Seo, Jungyun
    • Annual Conference on Human and Language Technology
    • /
    • 2018.10a
    • /
    • pp.243-246
    • /
    • 2018
  • 최근 사용자 발화를 이해하고 그에 맞는 피드백을 생성할 수 있는 대화 시스템의 중요성이 증가하고 있다. 따라서 사용자 의도를 파악하기 위한 화행 분석은 대화 시스템의 필수적인 요소이다. 최근 많이 연구되는 심층 학습 기법은 모델이 데이터로부터 자질들을 스스로 추출한다는 장점이 있다. 발화 자체의 연속성과 화자간 상호 작용을 포착하기 위하여 CNN에 RNN을 결합한 CNN-RNN을 제안한다. 본 논문에서 제안한 계층 구조 어텐션 매커니즘 기반 CNN-RNN을 효과적으로 적용한 결과 워드 임베딩을 추가한 조건에서 가장 높은 성능인 91.72% 정확도를 얻었다.

  • PDF

Blurred Image Enhancement Techniques Using Stack-Attention (Stack-Attention을 이용한 흐릿한 영상 강화 기법)

  • Park Chae Rim;Lee Kwang Ill;Cho Seok Je
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.2
    • /
    • pp.83-90
    • /
    • 2023
  • Blurred image is an important factor in lowering image recognition rates in Computer vision. This mainly occurs when the camera is unstablely out of focus or the object in the scene moves quickly during the exposure time. Blurred images greatly degrade visual quality, weakening visibility, and this phenomenon occurs frequently despite the continuous development digital camera technology. In this paper, it replace the modified building module based on the Deep multi-patch neural network designed with convolution neural networks to capture details of input images and Attention techniques to focus on objects in blurred images in many ways and strengthen the image. It measures and assigns each weight at different scales to differentiate the blurring of change and restores from rough to fine levels of the image to adjust both global and local region sequentially. Through this method, it show excellent results that recover degraded image quality, extract efficient object detection and features, and complement color constancy.