• Title/Summary/Keyword: Self-Attention

Search Result 1,170, Processing Time 0.029 seconds

A Study on the Factors Affecting Self-Concept of Children and Adolescents with Epilepsy (뇌전증 소아청소년 환아의 자아개념에 영향을 미치는 요인에 대한 연구)

  • Ha, Su Hee;Choi, Hee-Yeon;Lee, Hyang Woon;Kim, Eui-Jung
    • Journal of the Korean Academy of Child and Adolescent Psychiatry
    • /
    • v.28 no.4
    • /
    • pp.252-259
    • /
    • 2017
  • Objective: The purpose of this study was to investigate the impact of clinical and psychological factors on the self-concept of children and adolescents with epilepsy. Methods: Children and adolescents with epilepsy (n=60; age range=9-17 years) completed questionnaires about their epilepsy-related variables, self-concept, depressive symptoms, anxiety, family functions, and behavioral problems. The T-test and one-way analysis of variance were used to examine the variables affecting the total self-concept scores. To determine the independent variables by adjusting the significant variables, a stepwise regression analysis was performed. Results: In the correlational analysis, age, depressive symptoms, anxiety, social problems, attention problems, and internalizing problems had significantly negative correlations with self-concept. On the other hand, IQ and family functions showed positive correlations with selfconcept. Age (${\beta}=-0.177$, p=0.015), depressive symptoms (${\beta}=-0.487$, p<0.001), anxiety (${\beta}=-0.298$, p=0.008), and attention problems (${\beta}=-0.138$, p=0.048) were analyzed as independent factors to assess their impact on self-concept, and were found to account for 78.3% of the variance in self-concept by stepwise regression analysis. Conclusion: Parents and clinicians should pay attention to improving the self-concept of children and adolescents with epilepsy, especially if they have problems with depression, anxiety, or attention.

Relationship between self-efficacy and learning attitude according to smoking experience in the middle school students (일부 지역 중학생의 흡연경험에 따른 자기효능감과 학습태도의 관련성)

  • Son, Eun-Joo;Jang, Kyeung-Ae
    • Journal of Korean society of Dental Hygiene
    • /
    • v.15 no.5
    • /
    • pp.805-811
    • /
    • 2015
  • Objectives: The purpose of the study is to investigate the relationship between self-efficacy and learning attitude according to smoking experience in the middle school students. Methods: A self-reported questionnaire was completed by 608 middle school students in Gyeongnam from July 1 to 23, 2013. The questionnaire consisted of general characteristics of the subjects, smoking behavior, self-efficacy, and learning attitude. The questionnaire was adapted and modified from Kang, Park, and Koh. The self-efficacy was divided into general efficacy and social efficacy. The learning attitude was divided into attention concentration, learning method, and self learning. Data were analyzed using SPSS Win 21.0 program. Results: The nonsmoking students tended to have higher general efficacy and social efficacy than the smokers (p<0.01). The nonsmokers had more attention concentration in learning attitude than the smokers (p<0.001). The learning method (p<0.001) and self learning (p<0.001) showed the same results between the two groups. The smoking experience had the negative correlation with general efficacy (r=-0.164) and social efficacy(r=-0.154). The general efficacy is positively related to social efficacy (r=0.568). The smoking experience had the negative correlation to attention concentration (r=-0.235), learning method (r=-0.211) and self learning (r=-0.148). The attention concentration was positive relation with learning method (r=0.690) and self learning(r=0.662. The learning method had positive relation to self learning (r=0.764). Conclusions: The smoking students tended to have lower self-efficacy and learning attitude, so it is necessary to implement the smoking prevention program in the middle school students.

Influence of Socially-Prescribed Perfectionism on Social anxiety and Depression in Academic High School Students: Mediation Effects of Self-focused Attention and Self-Criticism (인문계 고등학생의 사회부과 완벽주의가 우울과 사회불안에 미치는 영향: 자기초점적 주의와 자기비난의 매개효과)

  • Kim, Seul-Ki;Lee, Dong-gwi
    • Korean Journal of School Psychology
    • /
    • v.15 no.2
    • /
    • pp.243-264
    • /
    • 2018
  • The study examined the influence of socially-prescribed perfectionism (SPP) on depression and social anxiety, and further investigated the mediating effects of self-focused attention and self-criticism. The questionnaires designed to measure multidimensional perfectionism, social anxiety, depression, self-focused attention, self-criticism scale for adolescents were administered twice at an interval of three weeks to 273 students (83 men, 190 women) enrolled at high schools in Gyeonggi-do Province. The findings for the present study were as follows. First, SPP, depression, social anxiety, self-focused attention, and self-criticism showed all positive correlations. Second, the mediation effect from the SPP to depression via self-focused attention was statistically significant, whereas the indirect effect from the SPP to depression via self-criticism was not. Third, the pattern in depression was the same in social anxiety. The results provide indirect support for the social anxiety cognitive model (Clark & Wells) with regards to social anxiety particularly in Korean high school students. Finally, the implications and limitations of this study and suggestions for future research were discussed.

An Effective Sentence Similarity Measure Method Based FAQ System Using Self-Attentive Sentence Embedding (Self-Attention 기반의 문장 임베딩을 이용한 효과적인 문장 유사도 기법 기반의 FAQ 시스템)

  • Kim, Bosung;Kim, Juae;Lee, Jeong-Eom;Kim, Seona;Ko, Youngjoong;Seo, Jungyun
    • Annual Conference on Human and Language Technology
    • /
    • 2018.10a
    • /
    • pp.361-363
    • /
    • 2018
  • FAQ 시스템은 주어진 질문과 가장 유사한 질의를 찾아 이에 대한 답을 제공하는 시스템이다. 질의 간의 유사도를 측정하기 위해 문장을 벡터로 표현하며 일반적으로 TFIDF, Okapi BM25와 같은 방법으로 계산한 단어 가중치 벡터를 이용하여 문장을 표현한다. 하지만 단어 가중치 벡터는 어휘적 정보를 표현하는데 유용한 반면 단어의 의미적인(semantic) 정보는 표현하기 어렵다. 본 논문에서는 이를 보완하고자 딥러닝을 이용한 문장 임베딩을 구축하고 단어 가중치 벡터와 문장 임베딩을 조합한 문장 유사도 계산 모델을 제안한다. 또한 문장 임베딩 구현 시 self-attention 기법을 적용하여 문장 내 중요한 부분에 가중치를 주었다. 실험 결과 제안하는 유사도 계산 모델은 비교 모델에 비해 모두 높은 성능을 보였고 self-attention을 적용한 실험에서는 추가적인 성능 향상이 있었다.

  • PDF

A study on skip-connection with time-frequency self-attention for improving speech enhancement based on complex-valued spectrum (복소 스펙트럼 기반 음성 향상의 성능 향상을 위한 time-frequency self-attention 기반 skip-connection 기법 연구)

  • Jaehee Jung;Wooil Kim
    • The Journal of the Acoustical Society of Korea
    • /
    • v.42 no.2
    • /
    • pp.94-101
    • /
    • 2023
  • A deep neural network composed of encoders and decoders, such as U-Net, used for speech enhancement, concatenates the encoder to the decoder through skip-connection. Skip-connection helps reconstruct the enhanced spectrum and complement the lost information. The features of the encoder and the decoder connected by the skip-connection are incompatible with each other. In this paper, for complex-valued spectrum based speech enhancement, Self-Attention (SA) method is applied to skip-connection to transform the feature of encoder to be compatible with the features of decoder. SA is a technique in which when generating an output sequence in a sequence-to-sequence tasks the weighted average of input is used to put attention on subsets of input, showing that noise can be effectively eliminated by being applied in speech enhancement. The three models using encoder and decoder features to apply SA to skip-connection are studied. As experimental results using TIMIT database, the proposed methods show improvements in all evaluation metrics compared to the Deep Complex U-Net (DCUNET) with skip-connection only.

Time-Series Forecasting Based on Multi-Layer Attention Architecture

  • Na Wang;Xianglian Zhao
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.1
    • /
    • pp.1-14
    • /
    • 2024
  • Time-series forecasting is extensively used in the actual world. Recent research has shown that Transformers with a self-attention mechanism at their core exhibit better performance when dealing with such problems. However, most of the existing Transformer models used for time series prediction use the traditional encoder-decoder architecture, which is complex and leads to low model processing efficiency, thus limiting the ability to mine deep time dependencies by increasing model depth. Secondly, the secondary computational complexity of the self-attention mechanism also increases computational overhead and reduces processing efficiency. To address these issues, the paper designs an efficient multi-layer attention-based time-series forecasting model. This model has the following characteristics: (i) It abandons the traditional encoder-decoder based Transformer architecture and constructs a time series prediction model based on multi-layer attention mechanism, improving the model's ability to mine deep time dependencies. (ii) A cross attention module based on cross attention mechanism was designed to enhance information exchange between historical and predictive sequences. (iii) Applying a recently proposed sparse attention mechanism to our model reduces computational overhead and improves processing efficiency. Experiments on multiple datasets have shown that our model can significantly increase the performance of current advanced Transformer methods in time series forecasting, including LogTrans, Reformer, and Informer.

A Multi-task Self-attention Model Using Pre-trained Language Models on Universal Dependency Annotations

  • Kim, Euhee
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.11
    • /
    • pp.39-46
    • /
    • 2022
  • In this paper, we propose a multi-task model that can simultaneously predict general-purpose tasks such as part-of-speech tagging, lemmatization, and dependency parsing using the UD Korean Kaist v2.3 corpus. The proposed model thus applies the self-attention technique of the BERT model and the graph-based Biaffine attention technique by fine-tuning the multilingual BERT and the two Korean-specific BERTs such as KR-BERT and KoBERT. The performances of the proposed model are compared and analyzed using the multilingual version of BERT and the two Korean-specific BERT language models.

De Novo Drug Design Using Self-Attention Based Variational Autoencoder (Self-Attention 기반의 변분 오토인코더를 활용한 신약 디자인)

  • Piao, Shengmin;Choi, Jonghwan;Seo, Sangmin;Kim, Kyeonghun;Park, Sanghyun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.11 no.1
    • /
    • pp.11-18
    • /
    • 2022
  • De novo drug design is the process of developing new drugs that can interact with biological targets such as protein receptors. Traditional process of de novo drug design consists of drug candidate discovery and drug development, but it requires a long time of more than 10 years to develop a new drug. Deep learning-based methods are being studied to shorten this period and efficiently find chemical compounds for new drug candidates. Many existing deep learning-based drug design models utilize recurrent neural networks to generate a chemical entity represented by SMILES strings, but due to the disadvantages of the recurrent networks, such as slow training speed and poor understanding of complex molecular formula rules, there is room for improvement. To overcome these shortcomings, we propose a deep learning model for SMILES string generation using variational autoencoders with self-attention mechanism. Our proposed model decreased the training time by 1/26 compared to the latest drug design model, as well as generated valid SMILES more effectively.

Effects of Parental Variables, Temperament and Internal Locus of Control on Self-Regulation of Children (부모요인과 아동의 기질 및 내재적 통제소재가 자기조절능력에 미치는 영향)

  • Lee, Kyung-Nim
    • Journal of Families and Better Life
    • /
    • v.28 no.6
    • /
    • pp.47-57
    • /
    • 2010
  • This study examines the effects of parental variable(parental support and supervision), temperament(activity level, attention span/persistence, and emotionality) and the internal locus of control on self-regulation of children. Data were collected from 455 5th and 6th graders and analyzed with Pearson's correlations and pathway analysis. The results were as follows : Children's temperament, internal locus of control and parental variable directly affected children's self-regulation. Parental variables mediated between children's temperament and internal locus of control and self-regulation. Internal locus of control mediated between children's temperament and self-regulation: in addition, the most important variable predicting children's self-regulation was children's attention span/persistence temperament.

Effects of Skipping Breakfast on Nutrition Status, Fatigue Level, and Attention Level among Middle School Students in Gyunggi Province, Korea (아침 결식이 경기지역 남녀 중학생의 영양섭취상태, 피로자각도 및 주의집중력에 미치는 영향)

  • Yim, Kyeong Sook
    • Journal of the Korean Society of Food Culture
    • /
    • v.29 no.5
    • /
    • pp.464-475
    • /
    • 2014
  • Eating breakfast provides crucial nutrition for brain function and helps promote overall health. It is especially critical in growing adolescents, as it is known to form good eating habits and better study habits. This study investigated the effects of skipping breakfast on nutritional state, fatigue level, and attention level. A cross-sectional study was conducted in 2010 on total of 828 adolescents composed of 414 boys and 414 girls. Students who ate breakfast never to twice per week were placed in the breakfast-skipper group while students who ate breakfast more than five times per week were included in the breakfast-eater group. Students performed a self-reported questionnaire on food behaviors, amount of food consumption, fatigue level, attention deficient hyperactivity disease (ADHD) level by Conners-Wells' Adolescent Self-Report Scales, depression scale, and self-esteem level. Statistical analysis was conducted using the SAS program (version 9.1). A total of 135 boys (32.6%) and 138 girls (33.3%) were included in the breakfast-skipper group, whereas 241 boys (58.2%) and 223 girls (53.9%) were included in the breakfast-eater group. The breakfast-skipper group showed irregular food behaviors and lacked nutrients. Specifically, energy (p< .001), protein (p< .001), dietary fiber (p< .001), calcium (p< .01), vitamin A (p< .01), thiamin (p< .05), niacin (p< .001) levels in boy breakfast-skippers were statistically lower compared to boy breakfast-eaters. Intakes of all nutrients except fat in girl breakfast-skippers were statistically lower than in girl breakfast-eaters. Girl breakfast-skippers (41.3%) showed significantly higher fatigue risks compared to girl breakfast-eaters (21.5%). Low attention level was also observed only in girls in the breakfast-skipping group. Moreover, students in the breakfast-skipper group showed higher scores for depression and low self-esteem (p< .001). In conclusion, skipping breakfast has effects on young adolescents' nutrition, manifesting as high fatigue level and low attention level, especially in girls.