• Title/Summary/Keyword: attention mechanism

Search Result 794, Processing Time 0.023 seconds

Study on Evaluation Method of Task-Specific Adaptive Differential Privacy Mechanism in Federated Learning Environment (연합 학습 환경에서의 Task-Specific Adaptive Differential Privacy 메커니즘 평가 방안 연구)

  • Assem Utaliyeva;Yoon-Ho Choi
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.34 no.1
    • /
    • pp.143-156
    • /
    • 2024
  • Federated Learning (FL) has emerged as a potent methodology for decentralized model training across multiple collaborators, eliminating the need for data sharing. Although FL is lauded for its capacity to preserve data privacy, it is not impervious to various types of privacy attacks. Differential Privacy (DP), recognized as the golden standard in privacy-preservation techniques, is widely employed to counteract these vulnerabilities. This paper makes a specific contribution by applying an existing, task-specific adaptive DP mechanism to the FL environment. Our comprehensive analysis evaluates the impact of this mechanism on the performance of a shared global model, with particular attention to varying data distribution and partitioning schemes. This study deepens the understanding of the complex interplay between privacy and utility in FL, providing a validated methodology for securing data without compromising performance.

Improved Anatomical Landmark Detection Using Attention Modules and Geometric Data Augmentation in X-ray Images (어텐션 모듈과 기하학적 데이터 증강을 통한 X-ray 영상 내 해부학적 랜드마크 검출 성능 향상)

  • Lee, Hyo-Jeong;Ma, Se-Rie;Choi, Jang-Hwan
    • Journal of the Korea Computer Graphics Society
    • /
    • v.28 no.3
    • /
    • pp.55-65
    • /
    • 2022
  • Recently, deep learning-based automated systems for identifying and detecting landmarks have been proposed. In order to train such a deep learning-based model without overfitting, a large amount of image and labeling data is required. Conventionally, an experienced reader manually identifies and labels landmarks in a patient's image. However, such measurement is not only expensive, but also has poor reproducibility, so the need for an automated labeling method has been raised. In addition, in the X-ray image, since various human tissues on the path through which the photons pass are displayed, it is difficult to identify the landmark compared to a general natural image or a 3D image modality image. In this study, we propose a geometric data augmentation technique that enables the generation of a large amount of labeling data in X-ray images. In addition, the optimal attention mechanism for landmark detection was presented through the implementation and application of various attention techniques to improve the detection performance of 16 major landmarks in the skull. Finally, among the major cranial landmarks, markers that ensure stable detection are derived, and these markers are expected to have high clinical application potential.

Prediction of Music Generation on Time Series Using Bi-LSTM Model (Bi-LSTM 모델을 이용한 음악 생성 시계열 예측)

  • Kwangjin, Kim;Chilwoo, Lee
    • Smart Media Journal
    • /
    • v.11 no.10
    • /
    • pp.65-75
    • /
    • 2022
  • Deep learning is used as a creative tool that could overcome the limitations of existing analysis models and generate various types of results such as text, image, and music. In this paper, we propose a method necessary to preprocess audio data using the Niko's MIDI Pack sound source file as a data set and to generate music using Bi-LSTM. Based on the generated root note, the hidden layers are composed of multi-layers to create a new note suitable for the musical composition, and an attention mechanism is applied to the output gate of the decoder to apply the weight of the factors that affect the data input from the encoder. Setting variables such as loss function and optimization method are applied as parameters for improving the LSTM model. The proposed model is a multi-channel Bi-LSTM with attention that applies notes pitch generated from separating treble clef and bass clef, length of notes, rests, length of rests, and chords to improve the efficiency and prediction of MIDI deep learning process. The results of the learning generate a sound that matches the development of music scale distinct from noise, and we are aiming to contribute to generating a harmonistic stable music.

Attention based Feature-Fusion Network for 3D Object Detection (3차원 객체 탐지를 위한 어텐션 기반 특징 융합 네트워크)

  • Sang-Hyun Ryoo;Dae-Yeol Kang;Seung-Jun Hwang;Sung-Jun Park;Joong-Hwan Baek
    • Journal of Advanced Navigation Technology
    • /
    • v.27 no.2
    • /
    • pp.190-196
    • /
    • 2023
  • Recently, following the development of LIDAR technology which can detect distance from the object, the interest for LIDAR based 3D object detection network is getting higher. Previous networks generate inaccurate localization results due to spatial information loss during voxelization and downsampling. In this study, we propose an attention-based convergence method and a camera-LIDAR convergence system to acquire high-level features and high positional accuracy. First, by introducing the attention method into the Voxel-RCNN structure, which is a grid-based 3D object detection network, the multi-scale sparse 3D convolution feature is effectively fused to improve the performance of 3D object detection. Additionally, we propose the late-fusion mechanism for fusing outcomes in 3D object detection network and 2D object detection network to delete false positive. Comparative experiments with existing algorithms are performed using the KITTI data set, which is widely used in the field of autonomous driving. The proposed method showed performance improvement in both 2D object detection on BEV and 3D object detection. In particular, the precision was improved by about 0.54% for the car moderate class compared to Voxel-RCNN.

Chart-based Stock Price Prediction by Combing Variation Autoencoder and Attention Mechanisms (변이형 오토인코더와 어텐션 메커니즘을 결합한 차트기반 주가 예측)

  • Sanghyun Bae;Byounggu Choi
    • Information Systems Review
    • /
    • v.23 no.1
    • /
    • pp.23-43
    • /
    • 2021
  • Recently, many studies have been conducted to increase the accuracy of stock price prediction by analyzing candlestick charts using artificial intelligence techniques. However, these studies failed to consider the time-series characteristics of candlestick charts and to take into account the emotional state of market participants in data learning for stock price prediction. In order to overcome these limitations, this study produced input data by combining volatility index and candlestick charts to consider the emotional state of market participants, and used the data as input for a new method proposed on the basis of combining variantion autoencoder (VAE) and attention mechanisms for considering the time-series characteristics of candlestick chart. Fifty firms were randomly selected from the S&P 500 index and their stock prices were predicted to evaluate the performance of the method compared with existing ones such as convolutional neural network (CNN) or long-short term memory (LSTM). The results indicated the method proposed in this study showed superior performance compared to the existing ones. This study implied that the accuracy of stock price prediction could be improved by considering the emotional state of market participants and the time-series characteristics of the candlestick chart.

Attention Based Collaborative Source-Side DDoS Attack Detection (어텐션 기반 협업형 소스측 분산 서비스 거부 공격 탐지)

  • Hwisoo Kim;Songheon Jeong;Kyungbaek Kim
    • The Transactions of the Korea Information Processing Society
    • /
    • v.13 no.4
    • /
    • pp.157-165
    • /
    • 2024
  • The evolution of the Distributed Denial of Service Attack(DDoS Attack) method has increased the difficulty in the detection process. One of the solutions to overcome the problems caused by the limitations of the existing victim-side detection method was the source-side detection technique. However, there was a problem of performance degradation due to network traffic irregularities. In order to solve this problem, research has been conducted to detect attacks using a collaborative network between several nodes based on artificial intelligence. Existing methods have shown limitations, especially in nonlinear traffic environments with high Burstness and jitter. To overcome this problem, this paper presents a collaborative source-side DDoS attack detection technique introduced with an attention mechanism. The proposed method aggregates detection results from multiple sources and assigns weights to each region, and through this, it is possible to effectively detect overall attacks and attacks in specific few areas. In particular, it shows a high detection rate with a low false positive of about 6% and a high detection rate of up to 4.3% in a nonlinear traffic dataset, and it can also confirm improvement in attack detection problems in a small number of regions compared to methods that showed limitations in the existing nonlinear traffic environment.

Bit-width Aware Generator and Intermediate Layer Knowledge Distillation using Channel-wise Attention for Generative Data-Free Quantization

  • Jae-Yong Baek;Du-Hwan Hur;Deok-Woong Kim;Yong-Sang Yoo;Hyuk-Jin Shin;Dae-Hyeon Park;Seung-Hwan Bae
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.7
    • /
    • pp.11-20
    • /
    • 2024
  • In this paper, we propose the BAG (Bit-width Aware Generator) and the Intermediate Layer Knowledge Distillation using Channel-wise Attention to reduce the knowledge gap between a quantized network, a full-precision network, and a generator in GDFQ (Generative Data-Free Quantization). Since the generator in GDFQ is only trained by the feedback from the full-precision network, the gap resulting in decreased capability due to low bit-width of the quantized network has no effect on training the generator. To alleviate this problem, BAG is quantized with same bit-width of the quantized network, and it can generate synthetic images, which are effectively used for training the quantized network. Typically, the knowledge gap between the quantized network and the full-precision network is also important. To resolve this, we compute channel-wise attention of outputs of convolutional layers, and minimize the loss function as the distance of them. As the result, the quantized network can learn which channels to focus on more from mimicking the full-precision network. To prove the efficiency of proposed methods, we quantize the network trained on CIFAR-100 with 3 bit-width weights and activations, and train it and the generator with our method. As the result, we achieve 56.14% Top-1 Accuracy and increase 3.4% higher accuracy compared to our baseline AdaDFQ.

LOW COST DEBRIS ANALYSIS FOR INDUSTRIAL MACHINERY CONDITION EVALUATION

  • Raadnui, S.
    • Proceedings of the Korean Society of Tribologists and Lubrication Engineers Conference
    • /
    • 2002.10b
    • /
    • pp.465-466
    • /
    • 2002
  • In any mechanical system consisting of gears, shafts and/or bearings, the majority of metallic particles deposited into and carried by the lubrication system originate from the deterioration of oil-wetted working surfaces, even in proper lubrication system, due to failure mechanism (s) such as wear, fatigue and fretting corrosion. Determination of the point at which transition from normal to abnormal or to actual damage occurs has become a focus of attention in research activities for years, because it has been recognized that reliable, economic operation can be achieved through appropriate preventative measures. Known collectively from 'all size wear debris analysis' as early failure detection, the methods of testing for damage differ considerably, range from a micron or a submicron size debris analysis to Magnetic Chip Detector (MCD) ferrous debris analysis. This paper will be focused on the utilization of the low-cost analysis techniques for evaluation of industrial machinery condition.

  • PDF

A Study on Application of Space Design Diagrams based on Program Interpretation - Focusing on the Projects by MVRDV and Ben van Berkel - (프로그램 해석을 기반으로 한 공간 디자인에 있어 다이어그램 적용에 관한 연구 - MVRDV와 벤 반 버클의 프로젝트를 중심으로 -)

  • Kim, Su-Mi;Kim, Moon-Duck;Jung, Sa-Hee
    • Journal of the Korean housing association
    • /
    • v.20 no.1
    • /
    • pp.113-122
    • /
    • 2009
  • In the work of contemporary spatialization, the meaning of program is being an important conceptual element beyond simple functional dimension. The design of this diagram allows to balance among changing programs, and it comes into the spotlight as a medium, which links the virtual to the reality as embodying the virtual, which is not embodied realistically, realistically through the design process. In other words, this method is attracting the public attention as a design tool which can concentrate on design process rather than a fixed tool. This study focuses on the role of balancing among mixed programs and the diagram as a design tool. Therefore, the purpose of this study is to analyze and research the work of spatialization, which apply to diagrams of contemporary designers, is to identify operating mechanism and then is to examine the characteristics and importance of them.

Condensation of Nano-Size Polymer Aggregates by Spin Drying

  • Ishikawa, Atsushi;Kawai, Akira
    • Journal of Adhesion and Interface
    • /
    • v.6 no.1
    • /
    • pp.7-10
    • /
    • 2005
  • Condensation control of nano-particles has become important in order to fabricate minute condensed structures. In this study, we focus our attention on condensation mechanism of polymer aggregates in a resist film. The polymer aggregate is structural component of a resist material which is used in lithography process. The condensation nature of polymer aggregates in the resist film surface is observed by using atomic force microscope (AFM). By using the AFM, the condensation of polymer aggregates can be observed clearly. The condensation of polymer aggregate strongly affects to precise fabrication of resist pattern below 100nm size. The interaction force among polymer aggregates can be analyzed based on Derjaguin approximation. We also discuss about condensation nature of polymer aggregates in the resist film surface with the help of micro sphere model.

  • PDF