• 제목/요약/키워드: data pre-processing

검색결과 800건 처리시간 0.025초

Practical issues in signal processing for structural flexibility identification

  • Zhang, J.;Zhou, Y.;Li, P.J.
    • Smart Structures and Systems
    • /
    • 제15권1호
    • /
    • pp.209-225
    • /
    • 2015
  • Compared to ambient vibration testing, impact testing has the merit to extract not only structural modal parameters but also structural flexibility. Therefore, structural deflections under any static load can be predicted from the identified results of the impact test data. In this article, a signal processing procedure for structural flexibility identification is first presented. Especially, practical issues in applying the proposed procedure for structural flexibility identification are investigated, which include sensitivity analyses of three pre-defined parameters required in the data pre-processing stage to investigate how they affect the accuracy of the identified structural flexibility. Finally, multiple-reference impact test data of a three-span reinforced concrete T-beam bridge are simulated by the FE analysis, and they are used as a benchmark structure to investigate the practical issues in the proposed signal processing procedure for structural flexibility identification.

BERT를 이용한 한국어 특허상담 기계독해 (Korean Machine Reading Comprehension for Patent Consultation Using BERT)

  • 민재옥;박진우;조유정;이봉건
    • 정보처리학회논문지:소프트웨어 및 데이터공학
    • /
    • 제9권4호
    • /
    • pp.145-152
    • /
    • 2020
  • 기계독해는(Machine reading comprehension) 사용자 질의와 관련된 문서를 기계가 이해한 후 정답을 추론하는 인공지능 자연어처리 태스크를 말하며, 이러한 기계독해는 챗봇과 같은 자동상담 서비스에 활용될 수 있다. 최근 자연어처리 분야에서 가장 높은 성능을 보이고 있는 BERT 언어모델은 대용량의 데이터를 pre-training 한 후에 각 자연어처리 태스크에 대해 fine-tuning하여 학습된 모델로 추론함으로써 문제를 해결하는 방식이다. 본 논문에서는 BERT기반 특허상담 기계독해 태스크를 위해 특허상담 데이터 셋을 구축하고 그 구축 방법을 소개하며, patent 코퍼스를 pre-training한 Patent-BERT 모델과 특허상담 모델학습에 적합한 언어처리 알고리즘을 추가함으로써 특허상담 기계독해 태스크의 성능을 향상시킬 수 있는 방안을 제안한다. 본 논문에서 제안한 방법을 사용하여 특허상담 질의에 대한 정답 결정에서 성능이 향상됨을 보였다.

기계학습 기반 저 복잡도 긴장 상태 분류 모델 (Design of Low Complexity Human Anxiety Classification Model based on Machine Learning)

  • 홍은재;박형곤
    • 전기학회논문지
    • /
    • 제66권9호
    • /
    • pp.1402-1408
    • /
    • 2017
  • Recently, services for personal biometric data analysis based on real-time monitoring systems has been increasing and many of them have focused on recognition of emotions. In this paper, we propose a classification model to classify anxiety emotion using biometric data actually collected from people. We propose to deploy the support vector machine to build a classification model. In order to improve the classification accuracy, we propose two data pre-processing procedures, which are normalization and data deletion. The proposed algorithms are actually implemented based on Real-time Traffic Flow Measurement structure, which consists of data collection module, data preprocessing module, and creating classification model module. Our experiment results show that the proposed classification model can infers anxiety emotions of people with the accuracy of 65.18%. Moreover, the proposed model with the proposed pre-processing techniques shows the improved accuracy, which is 78.77%. Therefore, we can conclude that the proposed classification model based on the pre-processing process can improve the classification accuracy with lower computation complexity.

Comparison of Pre-processed Brain Tumor MR Images Using Deep Learning Detection Algorithms

  • Kwon, Hee Jae;Lee, Gi Pyo;Kim, Young Jae;Kim, Kwang Gi
    • Journal of Multimedia Information System
    • /
    • 제8권2호
    • /
    • pp.79-84
    • /
    • 2021
  • Detecting brain tumors of different sizes is a challenging task. This study aimed to identify brain tumors using detection algorithms. Most studies in this area use segmentation; however, we utilized detection owing to its advantages. Data were obtained from 64 patients and 11,200 MR images. The deep learning model used was RetinaNet, which is based on ResNet152. The model learned three different types of pre-processing images: normal, general histogram equalization, and contrast-limited adaptive histogram equalization (CLAHE). The three types of images were compared to determine the pre-processing technique that exhibits the best performance in the deep learning algorithms. During pre-processing, we converted the MR images from DICOM to JPG format. Additionally, we regulated the window level and width. The model compared the pre-processed images to determine which images showed adequate performance; CLAHE showed the best performance, with a sensitivity of 81.79%. The RetinaNet model for detecting brain tumors through deep learning algorithms demonstrated satisfactory performance in finding lesions. In future, we plan to develop a new model for improving the detection performance using well-processed data. This study lays the groundwork for future detection technologies that can help doctors find lesions more easily in clinical tasks.

풍황 하중조건 데이터 자동생성화를 이용한 풍력터빈 하중해석의 효율 향상에 관한 연구 (Study on the efficiency improvement of wind turbine load analysis by using automatic generation for wind load condition data)

  • 안경민;임동수;이현주;최원호;이승구
    • 한국신재생에너지학회:학술대회논문집
    • /
    • 한국신재생에너지학회 2006년도 추계학술대회
    • /
    • pp.269-272
    • /
    • 2006
  • Load analysis software enables to design wind turbines effectively and exactly. In this paper, Bladed software developed by Garrad Hassan and Partners is used for load analysis. When using Bladed software, many time is requested to input data which is called by pre-processing. So in this paper, pre-processing Is automated by in-house software(BX) With this BX software, we can reduce the total time for pre-processing about 90%.

  • PDF

A BERT-Based Automatic Scoring Model of Korean Language Learners' Essay

  • Lee, Jung Hee;Park, Ji Su;Shon, Jin Gon
    • Journal of Information Processing Systems
    • /
    • 제18권2호
    • /
    • pp.282-291
    • /
    • 2022
  • This research applies a pre-trained bidirectional encoder representations from transformers (BERT) handwriting recognition model to predict foreign Korean-language learners' writing scores. A corpus of 586 answers to midterm and final exams written by foreign learners at the Intermediate 1 level was acquired and used for pre-training, resulting in consistent performance, even with small datasets. The test data were pre-processed and fine-tuned, and the results were calculated in the form of a score prediction. The difference between the prediction and actual score was then calculated. An accuracy of 95.8% was demonstrated, indicating that the prediction results were strong overall; hence, the tool is suitable for the automatic scoring of Korean written test answers, including grammatical errors, written by foreigners. These results are particularly meaningful in that the data included written language text produced by foreign learners, not native speakers.

복합 브로드캐스팅 환경에서 이동 트랜잭션 처리 (Mobile Transaction Processing in Hybrid Broadcasting Environment)

  • 김성석;양순옥
    • 한국정보과학회논문지:데이타베이스
    • /
    • 제31권4호
    • /
    • pp.422-431
    • /
    • 2004
  • 최근에 이동 컴퓨팅 환경에서 여러 데이타 전송 모델이 연구되고 있다. 특히 서버가 반복적으로 필요한 정보를 전파해주는 주기적 푸시 모델에 대한 연구가 활발히 진행되고 있다. 그러나 데이타 평균 대기 시간은 브로드캐스트 한 주기의 길이에 상당히 영향을 받으며, 또한 여러 사용자들간의 접근 데이타가 차이가 날 경우 응답시간에 상당히 나빠질 수 있다. 이 경우, 그 사용자들은 차라리 서버에게 명시적으로 데이타를 요청하기를 바랄 것이다. 이러한 두 가지 접근방식을 모두 지원하는 것을 복합 브로드캐스트라고 한다. 이 환경에서, 본 논문에서는 새로운 이동 트랜잭션 처리 알고리즘(O-PreH)을 개발하였다. 우선 서버가 관리하는 데이타는 주기적 브로드캐스트 방식으로 처리되는 Push_Data와 요구-처리방식으로 처리되는 Pull_Data로 나뉘어 진다. 즉, 사용자는 요구하는 데이타의 타입에 따라 접근하는 방식이 차이가 난다. 또한 서버는 이동 트랜잭션 일관성 유지를 돕기 위해 주기적으로 무효화 보고를 전송해준다. 만약 사용자가 무효화 보고에 의해 하나 이상의 충돌을 발견한다면, 일관성을 침해하지 않는 범위 내에서 그 충돌 순서를 결정한 후(pre-reordering) 나머지 연산들을 비관적으로 수행시킨다. 자세한 실험 과정을 거쳐 제안한 알고리즘의 성능 향상을 보였다.

지리정보시스템 기반의 상수관망 모델링 시스템 연구 (A Study on Water Network Modeling System Based Upon GIS)

  • 김준현;나탈리아 야꾸니나
    • 환경영향평가
    • /
    • 제19권3호
    • /
    • pp.315-321
    • /
    • 2010
  • ArcView and water network models have been integrated to develop the water network modeling system based upon GIS. To develop this system, pre, main, and post processing systems are required. GIS programming technique was adopted by using the ArcView's script language Avenue. The input data of models have been prepared by using the AutoCAD Map3D through the conversion of modeling input data to GIS data for A city. The modeling has been implemented by using EPANET, WaterCAD, InfoWorks. To develop the post processing system, the modeling results of the water network models have been analyzed by using GIS. During the application process of the developed system to B city with 300,000 population, main problems were found in the constructed GIS DB of that city. Thus, pilot study area of B city has been constructed, and pre-, main, and post-processing techniques were invented based upon GIS. Finally, the problems related to waterworks GIS projects in Korea were discussed and solutions were suggested.

Efficient Implementation of Single Error Correction and Double Error Detection Code with Check Bit Pre-computation for Memories

  • Cha, Sanguhn;Yoon, Hongil
    • JSTS:Journal of Semiconductor Technology and Science
    • /
    • 제12권4호
    • /
    • pp.418-425
    • /
    • 2012
  • In this paper, efficient implementation of error correction code (ECC) processing circuits based on single error correction and double error detection (SEC-DED) code with check bit pre-computation is proposed for memories. During the write operation of memory, check bit pre-computation eliminates the overall bits computation required to detect a double error, thereby reducing the complexity of the ECC processing circuits. In order to implement the ECC processing circuits using the check bit pre-computation more efficiently, the proper SEC-DED codes are proposed. The H-matrix of the proposed SEC-DED code is the same as that of the odd-weight-column code during the write operation and is designed by replacing 0's with 1's at the last row of the H-matrix of the odd-weight-column code during the read operation. When compared with a conventional implementation utilizing the odd-weight- column code, the implementation based on the proposed SEC-DED code with check bit pre-computation achieves reductions in the number of gates, latency, and power consumption of the ECC processing circuits by up to 9.3%, 18.4%, and 14.1% for 64 data bits in a word.

KIAPS 관측자료 처리시스템에서의 AMSU-A 위성자료 초기 전처리와 편향보정 모듈 개발 (Development of Pre-Processing and Bias Correction Modules for AMSU-A Satellite Data in the KIAPS Observation Processing System)

  • 이시혜;김주혜;강전호;전형욱
    • 대기
    • /
    • 제23권4호
    • /
    • pp.453-470
    • /
    • 2013
  • As a part of the KIAPS Observation Processing System (KOPS), we have developed the modules of satellite radiance data pre-processing and quality control, which include observation operators to interpolate model state variables into radiances in observation space. AMSU-A (Advanced Microwave Sounding Unit-A) level-1d radiance data have been extracted using the BUFR (Binary Universal Form for the Representation of meteorological data) decoder and a first guess has been calculated with RTTOV (Radiative Transfer for TIROS Operational Vertical Sounder) version 10.2. For initial quality checks, the pixels contaminated by large amounts of cloud liquid water, heavy precipitation, and sea ice have been removed. Channels for assimilation, rejection, or monitoring have been respectively selected for different surface types since the errors from the skin temperature are caused by inaccurate surface emissivity. Correcting the bias caused by errors in the instruments and radiative transfer model is crucial in radiance data pre-processing. We have developed bias correction modules in two steps based on 30-day innovation statistics (observed radiance minus background; O-B). The scan bias correction has been calculated individually for each channel, satellite, and scan position. Then a multiple linear regression of the scan-bias-corrected innovations with several predictors has been employed to correct the airmass bias.