• Title/Summary/Keyword: Neural network Transformer

Search Result 109, Processing Time 0.025 seconds

Fine-tuning Neural Network for Improving Video Classification Performance Using Vision Transformer (Vision Transformer를 활용한 비디오 분류 성능 향상을 위한 Fine-tuning 신경망)

  • Kwang-Yeob Lee;Ji-Won Lee;Tae-Ryong Park
    • Journal of IKEEE
    • /
    • v.27 no.3
    • /
    • pp.313-318
    • /
    • 2023
  • This paper proposes a neural network applying fine-tuning as a way to improve the performance of Video Classification based on Vision Transformer. Recently, the need for real-time video image analysis based on deep learning has emerged. Due to the characteristics of the existing CNN model used in Image Classification, it is difficult to analyze the association of consecutive frames. We want to find and solve the optimal model by comparing and analyzing the Vision Transformer and Non-local neural network models with the Attention mechanism. In addition, we propose an optimal fine-tuning neural network model by applying various methods of fine-tuning as a transfer learning method. The experiment trained the model with the UCF101 dataset and then verified the performance of the model by applying a transfer learning method to the UTA-RLDD dataset.

The Intelligent Control Algorithm of a Transformer Cooling System (변압기 냉각시스템의 지능제어알고리즘)

  • Han, Do-Young;Won, Jae-Young
    • Korean Journal of Air-Conditioning and Refrigeration Engineering
    • /
    • v.22 no.8
    • /
    • pp.515-522
    • /
    • 2010
  • In order to improve the efficiency of a transformer cooling system, the intelligent algorithm was developed. The intelligent algorithm is composed of a setpoint algorithm and a control algorithm. The setpoint algorithm was developed by the neural network, and the control algorithm was developed by the fuzzy logic. These algorithms were used for the control of a blower and an oil pump of the transformer cooling system. In order to analyse performances of these algorithms, the dynamic model of a transformer cooling system was used. Based on various performance tests, energy savings and stable controls of a transformer cooling system were observed. Therefore, control algorithms developed for this study may be effectively used for the control of a transformer cooling system.

Intelligent Diagnosis System for DGA Using Fuzzy Pattern Classification and Neural Network (퍼지 패턴 분류와 뉴럴 네트워크를 이용한 지능형 유중가스 판정 시스템)

  • Cho, Sung-Min;Kweon, Dong-Jin;Nam, Chang-Hyun;Kim, Jae-Chul
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.56 no.12
    • /
    • pp.2084-2090
    • /
    • 2007
  • The DGA (Dissolved Gases Analysis) technique has been widely using for fault diagnosis of the power transformers. Some electric power utility company establishes the criteria of DGA to improve reliability, because of difference of operation environment and design of power transformer. In this paper, we introduce intelligent diagnosis system for DGA result of KEPCO (Korea Electric Power Cooperation). This system can classify patterns type of gases ratio that frequently occurs in recent result of gases analysis using Fuzzy Inference. The classification of Patterns let us know that major causes of gases generation based on type of patterns. Finally, Neural Network based on patterns diagnose transformer. NN was trained using result data of DGA of actually faulted transformers recently. Result of intelligent diagnosis system is right well in comparison with actual inner inspection of transformers.

Expert System for Fault Diagnosis of Transformer

  • Kim, Jae-Chul;Jeon, Hee-Jong;Kong, Seong-Gon;Yoon, Yong-Han;Choi, Do-Hyuk;Jeon, Young-Jae
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.7 no.1
    • /
    • pp.45-53
    • /
    • 1997
  • This paper presents hybrid expert system for diagnosis of electric power transformer faults. The expert system diagnose and detect faults in oil-filled power transformers based on dissolved gas analysis. As the preprocessing stage, fuzzy information theory is used to manage the uncertainty in transformer fault diagnosis using dissolved gas analysis. The Kohonen neural network takes the interim results by applying fuzzy informations theory as inputs, and performs the transformer fault diagnosis. The Proposed system tested gas records of power transformers from Korea Electric Power Corporation to verify the diagnosis performance of transformer faults.

  • PDF

A Connectionist Expert System for Fault Diagnosis of Power System (전력계통 사고구간 판정을 위한 Commectionist Expert System)

  • 김광호;박종근
    • The Transactions of the Korean Institute of Electrical Engineers
    • /
    • v.41 no.4
    • /
    • pp.331-338
    • /
    • 1992
  • The application of Connectionist expert system using neural network to fault diagnosis of power system is presented and compared with rule-based expert system. Also, the merits of Connectionist model using neural network is presented. In this paper, the neural network for fault diagnosis is hierarchically composed by 3 neural network classes. The whole power system is divided into subsystems, the neural networks (Class II) which take charge of each subsystem and the neural network (Class III) which connects subsystems are composed. Every section of power system is classified into one of the typical sections which can be applied with same diagnosis rules, as line-section, bus-section, transformer-section. For each typical section, only one neural network (Class I) is composed. As the proposed model has hierarchical structure, the great reduction of learning structure is achieved. With parallel distributed processing, we show the possibility of on-line fault diagnosis.

  • PDF

Transformer Based Deep Learning Techniques for HVAC System Anomaly Detection (HVAC 시스템의 이상 탐지를 위한 Transformer 기반 딥러닝 기법)

  • Changjoon Park;Junhwi Park;Namjung Kim;Jaehyun Lee;Jeonghwan Gwak
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2024.01a
    • /
    • pp.47-48
    • /
    • 2024
  • Heating, Ventilating, and Air Conditioning(HVAC) 시스템은 난방(Heating), 환기(Ventilating), 공기조화(Air Conditioning)를 제공하는 공조시스템으로, 실내 환경의 온도, 습도 조절 및 지속적인 순환 및 여과를 통해 실내 공기 질을 개선한다. 이러한 HVAC 시스템에 이상이 생기는 경우 공기 여과율이 낮아지며, COVID-19와 같은 법정 감염병 예방에 취약해진다. 또한 장비의 과부하를 유발하여, 시스템의 효율성 저하 및 에너지 낭비를 불러올 수 있다. 따라서 본 논문에서는 HVAC 시스템의 이상 탐지 및 조기 조치를 위한 Transformer 기반 이상 탐지 기법의 적용을 제안한다. Transformer는 기존 시계열 데이터 처리를 위한 기법인 Recurrent Neural Network(RNN)기반 모델의 구조적 한계점을 극복함에 따라 Long Term Dependency 문제를 해결하고, 병렬처리를 통해 효율적인 Feature 추출이 가능하다. Transformer 모델이 HVAC 시스템의 이상 탐지에서 RNN 기반의 비교군 모델보다 약 1.31%의 향상을 보이며, Transformer 모델을 통한 HVAC의 이상 탐지에 효율적임을 확인하였다.

  • PDF

On-Line Estimation of Partial Discharge Location in Power Transformer

  • Yoon, Yong-Han;Kim, Jae-Chul;Chung, Chan-Soo;Kwak, Hee-Ro;Kweon, Dong-Jin
    • Journal of Electrical Engineering and information Science
    • /
    • v.1 no.2
    • /
    • pp.45-51
    • /
    • 1996
  • This paper presents a neural network approach for on-line estimation of partial discharge(PD) location using advanced correlation technique in power transformer. Ultrasonic sensors detect ultrasonic signals generated by a PD and the proposed method calculates time difference between the ultrasonic signals at each sensor pair using the cross-correlation technique applied by moving average and the Hamming window. The neural network takes distance difference as inputs converted from time difference, and estimates the PD location. Case studies showed that the proposed method using advanced correlation technique and a neural network estimated the PD location better than conventional methods.

  • PDF

TeGCN:Transformer-embedded Graph Neural Network for Thin-filer default prediction (TeGCN:씬파일러 신용평가를 위한 트랜스포머 임베딩 기반 그래프 신경망 구조 개발)

  • Seongsu Kim;Junho Bae;Juhyeon Lee;Heejoo Jung;Hee-Woong Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.3
    • /
    • pp.419-437
    • /
    • 2023
  • As the number of thin filers in Korea surpasses 12 million, there is a growing interest in enhancing the accuracy of assessing their credit default risk to generate additional revenue. Specifically, researchers are actively pursuing the development of default prediction models using machine learning and deep learning algorithms, in contrast to traditional statistical default prediction methods, which struggle to capture nonlinearity. Among these efforts, Graph Neural Network (GNN) architecture is noteworthy for predicting default in situations with limited data on thin filers. This is due to their ability to incorporate network information between borrowers alongside conventional credit-related data. However, prior research employing graph neural networks has faced limitations in effectively handling diverse categorical variables present in credit information. In this study, we introduce the Transformer embedded Graph Convolutional Network (TeGCN), which aims to address these limitations and enable effective default prediction for thin filers. TeGCN combines the TabTransformer, capable of extracting contextual information from categorical variables, with the Graph Convolutional Network, which captures network information between borrowers. Our TeGCN model surpasses the baseline model's performance across both the general borrower dataset and the thin filer dataset. Specially, our model performs outstanding results in thin filer default prediction. This study achieves high default prediction accuracy by a model structure tailored to characteristics of credit information containing numerous categorical variables, especially in the context of thin filers with limited data. Our study can contribute to resolving the financial exclusion issues faced by thin filers and facilitate additional revenue within the financial industry.

A Study on the Discriminate between Magnetizing Inrush and Internal Faults of Power Transformer by Artificial Neural Network (신경회로망에 의한 변압기의 여자돌입과 내부고장 판별에 관한 연구)

  • Park, Chul-Won;Cho, Phil-Hun;Shin, Myong-Chul;Yoon, Sug-Moo
    • Proceedings of the KIEE Conference
    • /
    • 1995.07b
    • /
    • pp.606-609
    • /
    • 1995
  • This paper presents discriminate between magnetizing inrush and internal faults of power transformer by artificial neural networks trained with preprocessing of fault discriminant. The proposed neural networks contain multi-layer perceptron using back-propagation learning algorithm with logistic sigmoid activation function. For this training and test, we used the relaying signals obtained from the EMTP simulation of model power system. It is shown that the proposed transformer protection system by neural networks never misoperated.

  • PDF

Low Lumination Image Enhancement with Transformer based Curve Learning

  • Yulin Cao;Chunyu Li;Guoqing Zhang;Yuhui Zheng
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.9
    • /
    • pp.2626-2641
    • /
    • 2024
  • Images taken in low lamination condition suffer from low contrast and loss of information. Low lumination image enhancement algorithms are required to improve the quality and broaden the applications of such images. In this study, we proposed a new Low lumination image enhancement architecture consisting of a transformer-based curve learning and an encoder-decoder-based texture enhancer. Considering the high effectiveness of curve matching, we constructed a transformer-based network to estimate the learnable curve for pixel mapping. Curve estimation requires global relationships that can be extracted through the transformer framework. To further improve the texture detail, we introduced an encoder-decoder network to extract local features and suppress the noise. Experiments on LOL and SID datasets showed that the proposed method not only has competitive performance compared to state-of-the-art techniques but also has great efficiency.