• Title/Summary/Keyword: Transformer

Search Result 4,282, Processing Time 0.031 seconds

Active Front End Rectifier Control of DC Distribution System Using Neural Network (신경회로망을 적용한 직류배전시스템의 AFE 정류기 제어에 관한 연구)

  • Kim, Seongwan;Jeon, Hyeonmin;Kim, Jongsu
    • Journal of the Korean Society of Marine Environment & Safety
    • /
    • v.27 no.7
    • /
    • pp.1124-1128
    • /
    • 2021
  • As regulations of emissions from ships become more stringent, electric propulsion systems have been increasingly used to solve this problem in vessels ranging from large merchant ships to small and medium-sized ships. Methods for improving the efficiency of the electric propulsion system include the improvement of power sources; the use of a system linked to environmentally friendly power sources, such as batteries, fuel cells, and solar power; and the development of hardware and control methodology for rectifiers, power conversion devices, and propulsion motors. The method using a phase-shifting transformer with diodes has been widely used for rectification. Power semiconductor devices with grid connection to an environmentally friendly power source using DC distribution, a variable speed power source, and the application of small and medium-sized electric propulsion systems have been developed. Accordingly, the demand for active front-end (AFE) rectifiers is increasing. In this study, a method using a neural network rather than a conventional proportional-integral controller was proposed to control the AFE rectifier. Tested controller data were used to design a neural network controller trained through MATLAB/Simulink. The neural network controller was applied to a rectification system designed using PSIM software. The results indicated the effectiveness of improving the waveform and power factor DC output stage according to the load variation. The proposed system can be applied as a rectification system for small and medium-sized environmentally friendly ships.

A Study of Pre-trained Language Models for Korean Language Generation (한국어 자연어생성에 적합한 사전훈련 언어모델 특성 연구)

  • Song, Minchae;Shin, Kyung-shik
    • Journal of Intelligence and Information Systems
    • /
    • v.28 no.4
    • /
    • pp.309-328
    • /
    • 2022
  • This study empirically analyzed a Korean pre-trained language models (PLMs) designed for natural language generation. The performance of two PLMs - BART and GPT - at the task of abstractive text summarization was compared. To investigate how performance depends on the characteristics of the inference data, ten different document types, containing six types of informational content and creation content, were considered. It was found that BART (which can both generate and understand natural language) performed better than GPT (which can only generate). Upon more detailed examination of the effect of inference data characteristics, the performance of GPT was found to be proportional to the length of the input text. However, even for the longest documents (with optimal GPT performance), BART still out-performed GPT, suggesting that the greatest influence on downstream performance is not the size of the training data or PLMs parameters but the structural suitability of the PLMs for the applied downstream task. The performance of different PLMs was also compared through analyzing parts of speech (POS) shares. BART's performance was inversely related to the proportion of prefixes, adjectives, adverbs and verbs but positively related to that of nouns. This result emphasizes the importance of taking the inference data's characteristics into account when fine-tuning a PLMs for its intended downstream task.

Cross-Lingual Style-Based Title Generation Using Multiple Adapters (다중 어댑터를 이용한 교차 언어 및 스타일 기반의 제목 생성)

  • Yo-Han Park;Yong-Seok Choi;Kong Joo Lee
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.8
    • /
    • pp.341-354
    • /
    • 2023
  • The title of a document is the brief summarization of the document. Readers can easily understand a document if we provide them with its title in their preferred styles and the languages. In this research, we propose a cross-lingual and style-based title generation model using multiple adapters. To train the model, we need a parallel corpus in several languages with different styles. It is quite difficult to construct this kind of parallel corpus; however, a monolingual title generation corpus of the same style can be built easily. Therefore, we apply a zero-shot strategy to generate a title in a different language and with a different style for an input document. A baseline model is Transformer consisting of an encoder and a decoder, pre-trained by several languages. The model is then equipped with multiple adapters for translation, languages, and styles. After the model learns a translation task from parallel corpus, it learns a title generation task from monolingual title generation corpus. When training the model with a task, we only activate an adapter that corresponds to the task. When generating a cross-lingual and style-based title, we only activate adapters that correspond to a target language and a target style. An experimental result shows that our proposed model is only as good as a pipeline model that first translates into a target language and then generates a title. There have been significant changes in natural language generation due to the emergence of large-scale language models. However, research to improve the performance of natural language generation using limited resources and limited data needs to continue. In this regard, this study seeks to explore the significance of such research.

Prompt engineering to improve the performance of teaching and learning materials Recommendation of Generative Artificial Intelligence

  • Soo-Hwan Lee;Ki-Sang Song
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.8
    • /
    • pp.195-204
    • /
    • 2023
  • In this study, prompt engineering that improves prompts was explored to improve the performance of teaching and learning materials recommendations using generative artificial intelligence such as GPT and Stable Diffusion. Picture materials were used as the types of teaching and learning materials. To explore the impact of the prompt composition, a Zero-Shot prompt, a prompt containing learning target grade information, a prompt containing learning goals, and a prompt containing both learning target grades and learning goals were designed to collect responses. The collected responses were embedded using Sentence Transformers, dimensionalized to t-SNE, and visualized, and then the relationship between prompts and responses was explored. In addition, each response was clustered using the k-means clustering algorithm, then the adjacent value of the widest cluster was selected as a representative value, imaged using Stable Diffusion, and evaluated by 30 elementary school teachers according to the criteria for evaluating teaching and learning materials. Thirty teachers judged that three of the four picture materials recommended were of educational value, and two of them could be used for actual classes. The prompt that recommended the most valuable picture material appeared as a prompt containing both the target grade and the learning goal.

Understanding of Generative Artificial Intelligence Based on Textual Data and Discussion for Its Application in Science Education (텍스트 기반 생성형 인공지능의 이해와 과학교육에서의 활용에 대한 논의)

  • Hunkoog Jho
    • Journal of The Korean Association For Science Education
    • /
    • v.43 no.3
    • /
    • pp.307-319
    • /
    • 2023
  • This study aims to explain the key concepts and principles of text-based generative artificial intelligence (AI) that has been receiving increasing interest and utilization, focusing on its application in science education. It also highlights the potential and limitations of utilizing generative AI in science education, providing insights for its implementation and research aspects. Recent advancements in generative AI, predominantly based on transformer models consisting of encoders and decoders, have shown remarkable progress through optimization of reinforcement learning and reward models using human feedback, as well as understanding context. Particularly, it can perform various functions such as writing, summarizing, keyword extraction, evaluation, and feedback based on the ability to understand various user questions and intents. It also offers practical utility in diagnosing learners and structuring educational content based on provided examples by educators. However, it is necessary to examine the concerns regarding the limitations of generative AI, including the potential for conveying inaccurate facts or knowledge, bias resulting from overconfidence, and uncertainties regarding its impact on user attitudes or emotions. Moreover, the responses provided by generative AI are probabilistic based on response data from many individuals, which raises concerns about limiting insightful and innovative thinking that may offer different perspectives or ideas. In light of these considerations, this study provides practical suggestions for the positive utilization of AI in science education.

Comparative analysis of the digital circuit designing ability of ChatGPT (ChatGPT을 활용한 디지털회로 설계 능력에 대한 비교 분석)

  • Kihun Nam
    • The Journal of the Convergence on Culture Technology
    • /
    • v.9 no.6
    • /
    • pp.967-971
    • /
    • 2023
  • Recently, a variety of AI-based platform services are available, and one of them is ChatGPT that processes a large quantity of data in the natural language and generates an answer after self-learning. ChatGPT can perform various tasks including software programming in the IT sector. Particularly, it may help generate a simple program and correct errors using C Language, which is a major programming language. Accordingly, it is expected that ChatGPT is capable of effectively using Verilog HDL, which is a hardware language created in C Language. Verilog HDL synthesis, however, is to generate imperative sentences in a logical circuit form and thus it needs to be verified whether the products are executed properly. In this paper, we aim to select small-scale logical circuits for ease of experimentation and to verify the results of circuits generated by ChatGPT and human-designed circuits. As to experimental environments, Xilinx ISE 14.7 was used for module modeling, and the xc3s1000 FPGA chip was used for module embodiment. Comparative analysis was performed on the use area and processing time of FPGA to compare the performance of ChatGPT products and Verilog HDL products.

A Pilot Study on Outpainting-powered Pet Pose Estimation (아웃페인팅 기반 반려동물 자세 추정에 관한 예비 연구)

  • Gyubin Lee;Youngchan Lee;Wonsang You
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.24 no.1
    • /
    • pp.69-75
    • /
    • 2023
  • In recent years, there has been a growing interest in deep learning-based animal pose estimation, especially in the areas of animal behavior analysis and healthcare. However, existing animal pose estimation techniques do not perform well when body parts are occluded or not present. In particular, the occlusion of dog tail or ear might lead to a significant degradation of performance in pet behavior and emotion recognition. In this paper, to solve this intractable problem, we propose a simple yet novel framework for pet pose estimation where pet pose is predicted on an outpainted image where some body parts hidden outside the input image are reconstructed by the image inpainting network preceding the pose estimation network, and we performed a preliminary study to test the feasibility of the proposed approach. We assessed CE-GAN and BAT-Fill for image outpainting, and evaluated SimpleBaseline for pet pose estimation. Our experimental results show that pet pose estimation on outpainted images generated using BAT-Fill outperforms the existing methods of pose estimation on outpainting-less input image.

Very Short- and Long-Term Prediction Method for Solar Power (초 장단기 통합 태양광 발전량 예측 기법)

  • Mun Seop Yun;Se Ryung Lim;Han Seung Jang
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.18 no.6
    • /
    • pp.1143-1150
    • /
    • 2023
  • The global climate crisis and the implementation of low-carbon policies have led to a growing interest in renewable energy and a growing number of related industries. Among them, solar power is attracting attention as a representative eco-friendly energy that does not deplete and does not emit pollutants or greenhouse gases. As a result, the supplement of solar power facility is increasing all over the world. However, solar power is easily affected by the environment such as geography and weather, so accurate solar power forecast is important for stable operation and efficient management. However, it is very hard to predict the exact amount of solar power using statistical methods. In addition, the conventional prediction methods have focused on only short- or long-term prediction, which causes to take long time to obtain various prediction models with different prediction horizons. Therefore, this study utilizes a many-to-many structure of a recurrent neural network (RNN) to integrate short-term and long-term predictions of solar power generation. We compare various RNN-based very short- and long-term prediction methods for solar power in terms of MSE and R2 values.

Comparative Analysis of Self-supervised Deephashing Models for Efficient Image Retrieval System (효율적인 이미지 검색 시스템을 위한 자기 감독 딥해싱 모델의 비교 분석)

  • Kim Soo In;Jeon Young Jin;Lee Sang Bum;Kim Won Gyum
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.12
    • /
    • pp.519-524
    • /
    • 2023
  • In hashing-based image retrieval, the hash code of a manipulated image is different from the original image, making it difficult to search for the same image. This paper proposes and evaluates a self-supervised deephashing model that generates perceptual hash codes from feature information such as texture, shape, and color of images. The comparison models are autoencoder-based variational inference models, but the encoder is designed with a fully connected layer, convolutional neural network, and transformer modules. The proposed model is a variational inference model that includes a SimAM module of extracting geometric patterns and positional relationships within images. The SimAM module can learn latent vectors highlighting objects or local regions through an energy function using the activation values of neurons and surrounding neurons. The proposed method is a representation learning model that can generate low-dimensional latent vectors from high-dimensional input images, and the latent vectors are binarized into distinguishable hash code. From the experimental results on public datasets such as CIFAR-10, ImageNet, and NUS-WIDE, the proposed model is superior to the comparative model and analyzed to have equivalent performance to the supervised learning-based deephashing model. The proposed model can be used in application systems that require low-dimensional representation of images, such as image search or copyright image determination.

A Predictive Bearing Anomaly Detection Model Using the SWT-SVD Preprocessing Algorithm (SWT-SVD 전처리 알고리즘을 적용한 예측적 베어링 이상탐지 모델)

  • So-hyang Bak;Kwanghoon Pio Kim
    • Journal of Internet Computing and Services
    • /
    • v.25 no.1
    • /
    • pp.109-121
    • /
    • 2024
  • In various manufacturing processes such as textiles and automobiles, when equipment breaks down or stops, the machines do not work, which leads to time and financial losses for the company. Therefore, it is important to detect equipment abnormalities in advance so that equipment failures can be predicted and repaired before they occur. Most equipment failures are caused by bearing failures, which are essential parts of equipment, and detection bearing anomaly is the essence of PHM(Prognostics and Health Management) research. In this paper, we propose a preprocessing algorithm called SWT-SVD, which analyzes vibration signals from bearings and apply it to an anomaly transformer, one of the time series anomaly detection model networks, to implement bearing anomaly detection model. Vibration signals from the bearing manufacturing process contain noise due to the real-time generation of sensor values. To reduce noise in vibration signals, we use the Stationary Wavelet Transform to extract frequency components and perform preprocessing to extract meaningful features through the Singular Value Decomposition algorithm. For experimental validation of the proposed SWT-SVD preprocessing method in the bearing anomaly detection model, we utilize the PHM-2012-Challenge dataset provided by the IEEE PHM Conference. The experimental results demonstrate significant performance with an accuracy of 0.98 and an F1-Score of 0.97. Additionally, to substantiate performance improvement, we conduct a comparative analysis with previous studies, confirming that the proposed preprocessing method outperforms previous preprocessing methods in terms of performance.