• Title/Summary/Keyword: Transformer Model

Search Result 594, Processing Time 0.022 seconds

Unsupervised Abstractive Summarization Method that Suitable for Documents with Flows (흐름이 있는 문서에 적합한 비지도학습 추상 요약 방법)

  • Lee, Hoon-suk;An, Soon-hong;Kim, Seung-hoon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.10 no.11
    • /
    • pp.501-512
    • /
    • 2021
  • Recently, a breakthrough has been made in the NLP area by Transformer techniques based on encoder-decoder. However, this only can be used in mainstream languages where millions of dataset are well-equipped, such as English and Chinese, and there is a limitation that it cannot be used in non-mainstream languages where dataset are not established. In addition, there is a deflection problem that focuses on the beginning of the document in mechanical summarization. Therefore, these methods are not suitable for documents with flows such as fairy tales and novels. In this paper, we propose a hybrid summarization method that does not require a dataset and improves the deflection problem using GAN with two adaptive discriminators. We evaluate our model on the CNN/Daily Mail dataset to verify an objective validity. Also, we proved that the model has valid performance in Korean, one of the non-mainstream languages.

Korean Text Image Super-Resolution for Improving Text Recognition Accuracy (텍스트 인식률 개선을 위한 한글 텍스트 이미지 초해상화)

  • Junhyeong Kwon;Nam Ik Cho
    • Journal of Broadcast Engineering
    • /
    • v.28 no.2
    • /
    • pp.178-184
    • /
    • 2023
  • Finding texts in general scene images and recognizing their contents is a very important task that can be used as a basis for robot vision, visual assistance, and so on. However, for the low-resolution text images, the degradations, such as noise or blur included in text images, are more noticeable, which leads to severe performance degradation of text recognition accuracy. In this paper, we propose a new Korean text image super-resolution based on a Transformer-based model, which generally shows higher performance than convolutional neural networks. In the experiments, we show that text recognition accuracy for Korean text images can be improved when our proposed text image super-resolution method is used. We also propose a new Korean text image dataset for training our model, which contains massive HR-LR Korean text image pairs.

Study on the Application of RT-DETR to Monitoring of Coastal Debris on Unmanaged Coasts (비관리 해변의 해안 쓰레기 모니터링을 위한 RT-DETR 적용 방안 연구)

  • Ye-Been Do;Hong-Joo Yoon
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.19 no.2
    • /
    • pp.453-466
    • /
    • 2024
  • To improve the monitoring of Coastal Debris in the South Korea, which is difficult to estimate due to limited resources and vertex-based surveys, an approach based on UAV(Unmanned Aerial Vehicle) images and the RT-DETR(Realtime DEtection TRansformer) model was proposed for detecting Coastal Debris. By comparing to field investigation, the study suggested the possibility of quantitatively detecting coastal garbage and estimating the total capacity of garbage deposited on the natural coastline of the South Korea. The RT-DETR model achieved an accuracy of 0.894 for mAP@0.5 and 0.693 for mAP@0.5:0.95 in training. When applied to unmanaged coasts, the accuracy for the total number of coastal debris items was 72.9%. It is anticipated that if guidelines for defining monitoring of unmanaged coasts are established alongside this research, it should be possible to estimate the total capacity of the deposited coastal debris in the South Korea.

A high-density gamma white spots-Gaussian mixture noise removal method for neutron images denoising based on Swin Transformer UNet and Monte Carlo calculation

  • Di Zhang;Guomin Sun;Zihui Yang;Jie Yu
    • Nuclear Engineering and Technology
    • /
    • v.56 no.2
    • /
    • pp.715-727
    • /
    • 2024
  • During fast neutron imaging, besides the dark current noise and readout noise of the CCD camera, the main noise in fast neutron imaging comes from high-energy gamma rays generated by neutron nuclear reactions in and around the experimental setup. These high-energy gamma rays result in the presence of high-density gamma white spots (GWS) in the fast neutron image. Due to the microscopic quantum characteristics of the neutron beam itself and environmental scattering effects, fast neutron images typically exhibit a mixture of Gaussian noise. Existing denoising methods in neutron images are difficult to handle when dealing with a mixture of GWS and Gaussian noise. Herein we put forward a deep learning approach based on the Swin Transformer UNet (SUNet) model to remove high-density GWS-Gaussian mixture noise from fast neutron images. The improved denoising model utilizes a customized loss function for training, which combines perceptual loss and mean squared error loss to avoid grid-like artifacts caused by using a single perceptual loss. To address the high cost of acquiring real fast neutron images, this study introduces Monte Carlo method to simulate noise data with GWS characteristics by computing the interaction between gamma rays and sensors based on the principle of GWS generation. Ultimately, the experimental scenarios involving simulated neutron noise images and real fast neutron images demonstrate that the proposed method not only improves the quality and signal-to-noise ratio of fast neutron images but also preserves the details of the original images during denoising.

Simplified 2-D Analytical Model for Winding Loss Analysis of Flyback Transformers

  • Zhang, Junming;Yuan, Wei;Zeng, Hulong;Qian, Zhaoming
    • Journal of Power Electronics
    • /
    • v.12 no.6
    • /
    • pp.960-973
    • /
    • 2012
  • The winding loss analysis of a flyback transformer is difficult and ambiguous because the primary side current and the secondary side current differs both in shape and phase, especially for DCM (Discontinuous Conduction Mode) operation. Meanwhile, the fringing field caused by the air gaps further makes the traditional 1-D loss analysis model not directly applicable. The paper gives a thorough investigation into the phase shift of winding currents, which indicates that the phase shift of the high order harmonics is still close to $180^{\circ}$ out-of-phase. Based on the analysis, a simplified 2-D winding loss analytical model for flyback transformers considering the effects of low order harmonics is proposed. By neglecting the y components of the fringing field, the proposed model has an acceptable accuracy and a simple form that is similar to the conventional 1-D model. The power loss calculated with the proposed analysis model is verified by FEA (Finite Element Analysis) simulations and experimental results.

Efficiency Optimization with a Novel Magnetic-Circuit Model for Inductive Power Transfer in EVs

  • Tang, Yunyu;Zhu, Fan;Ma, Hao
    • Journal of Power Electronics
    • /
    • v.18 no.1
    • /
    • pp.309-322
    • /
    • 2018
  • The technology of inductive power transfer has been proved to be a promising solution in many applications especially in electric vehicle (EV) charging systems, due to its features of safety and convenience. However, loosely coupled transformers lead to the system efficiency not coming up to the expectation at the present time. Therefore, at first, the magnetic core losses are calculated with a novel magnetic-circuit model instead of the commonly used finite-element-method (FEM) simulations. The parameters in the model can be obtained with a one-time FEM simulation, which makes the calculation process expeditious. When compared with traditional methods, the model proposed in the paper is much less time-consuming and relatively accurate. These merits have been verified by experimental results. Furthermore, with the proposed loss calculation model, the system is optimized by parameter sweeping, such as the operating frequency and winding turns. Specifically, rather than a predesigned switching frequency, a more efficiency-optimized frequency for the series-parallel (SP) compensation topology is detected and a detailed investigation has been presented accordingly. The optimized system is capable of an efficiency that is greater than 93% at a coil separation distance of 200mm and coil dimensions of $600mm{\times}400mm$.

Structural reliability analysis using temporal deep learning-based model and importance sampling

  • Nguyen, Truong-Thang;Dang, Viet-Hung
    • Structural Engineering and Mechanics
    • /
    • v.84 no.3
    • /
    • pp.323-335
    • /
    • 2022
  • The main idea of the framework is to seamlessly combine a reasonably accurate and fast surrogate model with the importance sampling strategy. Developing a surrogate model for predicting structures' dynamic responses is challenging because it involves high-dimensional inputs and outputs. For this purpose, a novel surrogate model based on cutting-edge deep learning architectures specialized for capturing temporal relationships within time-series data, namely Long-Short term memory layer and Transformer layer, is designed. After being properly trained, the surrogate model could be utilized in place of the finite element method to evaluate structures' responses without requiring any specialized software. On the other hand, the importance sampling is adopted to reduce the number of calculations required when computing the failure probability by drawing more relevant samples near critical areas. Thanks to the portability of the trained surrogate model, one can integrate the latter with the Importance sampling in a straightforward fashion, forming an efficient framework called TTIS, which represents double advantages: less number of calculations is needed, and the computational time of each calculation is significantly reduced. The proposed approach's applicability and efficiency are demonstrated through three examples with increasing complexity, involving a 1D beam, a 2D frame, and a 3D building structure. The results show that compared to the conventional Monte Carlo simulation, the proposed method can provide highly similar reliability results with a reduction of up to four orders of magnitudes in time complexity.

Semantic Segmentation of the Habitats of Ecklonia Cava and Sargassum in Undersea Images Using HRNet-OCR and Swin-L Models (HRNet-OCR과 Swin-L 모델을 이용한 조식동물 서식지 수중영상의 의미론적 분할)

  • Kim, Hyungwoo;Jang, Seonwoong;Bak, Suho;Gong, Shinwoo;Kwak, Jiwoo;Kim, Jinsoo;Lee, Yangwon
    • Korean Journal of Remote Sensing
    • /
    • v.38 no.5_3
    • /
    • pp.913-924
    • /
    • 2022
  • In this paper, we presented a database construction of undersea images for the Habitats of Ecklonia cava and Sargassum and conducted an experiment for semantic segmentation using state-of-the-art (SOTA) models such as High Resolution Network-Object Contextual Representation (HRNet-OCR) and Shifted Windows-L (Swin-L). The result showed that our segmentation models were superior to the existing experiments in terms of the 29% increased mean intersection over union (mIOU). Swin-L model produced better performance for every class. In particular, the information of the Ecklonia cava class that had small data were also appropriately extracted by Swin-L model. Target objects and the backgrounds were well distinguished owing to the Transformer backbone better than the legacy models. A bigger database under construction will ensure more accuracy improvement and can be utilized as deep learning database for undersea images.

A Study of Fine Tuning Pre-Trained Korean BERT for Question Answering Performance Development (사전 학습된 한국어 BERT의 전이학습을 통한 한국어 기계독해 성능개선에 관한 연구)

  • Lee, Chi Hoon;Lee, Yeon Ji;Lee, Dong Hee
    • Journal of Information Technology Services
    • /
    • v.19 no.5
    • /
    • pp.83-91
    • /
    • 2020
  • Language Models such as BERT has been an important factor of deep learning-based natural language processing. Pre-training the transformer-based language models would be computationally expensive since they are consist of deep and broad architecture and layers using an attention mechanism and also require huge amount of data to train. Hence, it became mandatory to do fine-tuning large pre-trained language models which are trained by Google or some companies can afford the resources and cost. There are various techniques for fine tuning the language models and this paper examines three techniques, which are data augmentation, tuning the hyper paramters and partly re-constructing the neural networks. For data augmentation, we use no-answer augmentation and back-translation method. Also, some useful combinations of hyper parameters are observed by conducting a number of experiments. Finally, we have GRU, LSTM networks to boost our model performance with adding those networks to BERT pre-trained model. We do fine-tuning the pre-trained korean-based language model through the methods mentioned above and push the F1 score from baseline up to 89.66. Moreover, some failure attempts give us important lessons and tell us the further direction in a good way.

PSCAD/EMTDC Simulation Model of Variable Speed Wind Power Generation System Using Permanent Magnet Synchronous Machine (영구자석형 동기기에 의한 가변속 풍력발전 시스템의 PSCAD/EMTDC 시뮬레이션 모델 개발)

  • Kim Jeong-Jae;Song Seung-Ho
    • The Transactions of the Korean Institute of Power Electronics
    • /
    • v.10 no.6
    • /
    • pp.610-617
    • /
    • 2005
  • A variable speed wind turbine simulation model for grid connection is developed based on PSCAD/EMTDC. The model consists of wind model, rotor dynamics, synchronous generator, power converter, transformer, distribution line and infinite bus. Implementation of blade characteristics and power converter control strategies are included. Several transient case studies are performed including wind speed change, local load change and grid-side voltage unbalance using developed simulation model. The results of this work can be utilized for study of actual interaction between wind turbine and grid for reliable operation and protection of power system.