• Title/Summary/Keyword: data pre-processing

Search Result 801, Processing Time 0.031 seconds

Feasibility Study for an Optical Sensing System for Hardy Kiwi (Actinidia arguta) Sugar Content Estimation

  • Lee, Sangyoon;Sarkar, Shagor;Park, Youngki;Yang, Jaekyeong;Kweon, Giyoung
    • Journal of agriculture & life science
    • /
    • v.53 no.3
    • /
    • pp.147-157
    • /
    • 2019
  • In this study, we tried to find out the most appropriate pre-processing method and to verify the feasibility of developing a low-price sensing system for predicting the hardy kiwis sugar content based on VNIRS and subsequent spectral analysis. A total of 495 hardy kiwi samples were collected from three farms in Muju, Jeollabukdo, South Korea. The samples were scanned with a spectrophotometer in the range of 730-2300 nm with 1 nm spectral sampling interval. The measured data were arbitrarily separated into calibration and validation data for sugar content prediction. Partial least squares (PLS) regression was performed using various combinations of pre-processing methods. When the latent variable (LV) was 8 with the pre-processing combination of standard normal variate (SNV) and orthogonal signal correction (OSC), the highest R2 values of calibration and validation were 0.78 and 0.84, respectively. The possibility of predicting the sugar content of hardy kiwi was also examined at spectral sampling intervals of 6 and 10 nm in the narrower spectral range from 730 nm to 1200 nm for a low-price optical sensing system. The prediction performance had promising results with R2 values of 0.84 and 0.80 for 6 and 10 nm, respectively. Future studies will aim to develop a low-price optical sensing system with a combination of optical components such as photodiodes, light-emitting diodes (LEDs) and/or lamps, and to locate a more reliable prediction model by including meteorological data, soil data, and different varieties of hardy kiwi plants.

Design of Heavy Rain Advisory Decision Model Based on Optimized RBFNNs Using KLAPS Reanalysis Data (KLAPS 재분석 자료를 이용한 진화최적화 RBFNNs 기반 호우특보 판별 모델 설계)

  • Kim, Hyun-Myung;Oh, Sung-Kwun;Lee, Yong-Hee
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.23 no.5
    • /
    • pp.473-478
    • /
    • 2013
  • In this paper, we develop the Heavy Rain Advisory Decision Model based on intelligent neuro-fuzzy algorithm RBFNNs by using KLAPS(Korea Local Analysis and Prediction System) Reanalysis data. the prediction ability of existing heavy rainfall forecasting systems is usually affected by the processing techniques of meteorological data. In this study, we introduce the heavy rain forecast method using the pre-processing techniques of meteorological data are in order to improve these drawbacks of conventional system. The pre-processing techniques of meteorological data are designed by using point conversion, cumulative precipitation generation, time series data processing and heavy rain warning extraction methods based on KLAPS data. Finally, the proposed system forecasts cumulative rainfall for six hours after future t(t=1,2,3) hours and offers information to determine heavy rain advisory. The essential parameters of the proposed model such as polynomial order, the number of rules, and fuzzification coefficient are optimized by means of Differential Evolution.

Bayesian Estimation based K-1 Gas-Mask Shelf Life Assessment using CSRP Test Data (CSRP 시험데이터를 사용한 베이시안 추정모델 기반 K-1 방독면 저장수명 분석)

  • Kim, Jong-Hwan;Jung, Chi-jung;Kim, Hyunjung
    • Journal of the Korea Institute of Military Science and Technology
    • /
    • v.21 no.1
    • /
    • pp.124-132
    • /
    • 2018
  • This paper presents a shelf life assessment for K-1 military gas masks in the Republic of Korea using test data of Chemical Materiels Stockpile Reliability Program(CSRP). For the shelf life assessment, over 2,500 samples between 2006 and 2015 were collected from field tests and analyzed to estimate a probability of proper and improper functionality using Bayesian estimation. For this, three stages were considered; a pre-processing, a processing and an assessment. In the pre-processing, major components which directly influence the shelf life of the mask were statistically analyzed and selected by applying principal component analysis from all test components. In the processing, with the major components chosen in the previous stage, both proper and improper probability of gas masks were computed by applying Bayesian estimation. In the assessment, the probability model of the mask shelf life was analyzed with respect to storage periods between 0 and 29 years resulting in between 66.1 % and 100 % performances in accuracy, sensitivity, positive predictive value, and negative predictive value.

High Speed Character Recognition by Multiprocessor System (멀티 프로세서 시스템에 의한 고속 문자인식)

  • 최동혁;류성원;최성남;김학수;이용균;박규태
    • Journal of the Korean Institute of Telematics and Electronics B
    • /
    • v.30B no.2
    • /
    • pp.8-18
    • /
    • 1993
  • A multi-font, multi-size and high speed character recognition system is designed. The design principles are simpilcity of algorithm, adaptibility, learnability, hierachical data processing and attention by feed back. For the multi-size character recognition, the extracted character images are normalized. A hierachical classifier classifies the feature vectors. Feature is extracted by applying the directional receptive field after the directional dege filter processing. The hierachical classifier is consist of two pre-classifiers and one decision making classifier. The effect of two pre-classifiers is prediction to the final decision making classifier. With the pre-classifiers, the time to compute the distance of the final classifier is reduced. Recognition rate is 95% for the three documents printed in three kinds of fonts, total 1,700 characters. For high speed implemention, a multiprocessor system with the ring structure of four transputers is implemented, and the recognition speed of 30 characters per second is aquired.

  • PDF

Algorithm for Improving GPS Performance by Data Pre-processing (데이터 사전처리에 의한 GPS 성능 개선 알고리즘)

  • Rhee Jae-Hoon;Hong Won-Chul;Kim Hyun-Soo;Jeon Chang-Wan
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.12 no.8
    • /
    • pp.752-758
    • /
    • 2006
  • A GPS receiver provides much information such as calculated position, speed, heading, status of satellites, current time errors, etc. It is well-known that GPS signals from GPS receiver mounted on moving vehicle are often distorted, contaminated by various noises, and blocked by tunnel or tall buildings. The phenomenon often obstructs correct navigation especially when a vehicle keeps stopping or is moving in low speed. Therefore it is needed to pre-process the signals to adapt it to various applications. In this paper, an algorithm to pre-process the signals is proposed. For this, GPS data obtaining from uNAV GPS receiver are analyzed and classified based on dynamic characteristic. Then, the proposed algorithm is applied to the data and some test results are shown to verify the usefulness of the algorithm.

A Study of Fine Tuning Pre-Trained Korean BERT for Question Answering Performance Development (사전 학습된 한국어 BERT의 전이학습을 통한 한국어 기계독해 성능개선에 관한 연구)

  • Lee, Chi Hoon;Lee, Yeon Ji;Lee, Dong Hee
    • Journal of Information Technology Services
    • /
    • v.19 no.5
    • /
    • pp.83-91
    • /
    • 2020
  • Language Models such as BERT has been an important factor of deep learning-based natural language processing. Pre-training the transformer-based language models would be computationally expensive since they are consist of deep and broad architecture and layers using an attention mechanism and also require huge amount of data to train. Hence, it became mandatory to do fine-tuning large pre-trained language models which are trained by Google or some companies can afford the resources and cost. There are various techniques for fine tuning the language models and this paper examines three techniques, which are data augmentation, tuning the hyper paramters and partly re-constructing the neural networks. For data augmentation, we use no-answer augmentation and back-translation method. Also, some useful combinations of hyper parameters are observed by conducting a number of experiments. Finally, we have GRU, LSTM networks to boost our model performance with adding those networks to BERT pre-trained model. We do fine-tuning the pre-trained korean-based language model through the methods mentioned above and push the F1 score from baseline up to 89.66. Moreover, some failure attempts give us important lessons and tell us the further direction in a good way.

PreBAC: a novel Access Control scheme based Proxy Re-Encryption for cloud computing

  • Su, Mang;Wang, Liangchen
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.5
    • /
    • pp.2754-2767
    • /
    • 2019
  • Cloud computing is widely used in information spreading and processing, which has provided a easy and quick way for users to access data and retrieve service. Generally, in order to prevent the leakage of the information, the data in cloud is transferred in the encrypted form. As one of the traditional security technologies, access control is an important part for cloud security. However, the current access control schemes are not suitable for cloud, thus, it is a vital problem to design an access control scheme which should take account of complex factors to satisfy the various requirements for cipher text protection. We present a novel access control scheme based on proxy re-encryption(PRE) technology (PreBAC) for cipher text. It will suitable for the protection of data confidently and information privacy. At first, We will give the motivations and related works, and then specify system model for our scheme. Secondly, the algorithms are given and security of our scheme is proved. Finally, the comparisons between other schemes are made to show the advantages of PreBAC.

Development of a 1-Chip Application-Specific DSP for the Next Generation FAX Image Processing (차세대 팩스 영상처리를 위한 1-Chip Application-Specific DSP 기법)

  • 김재호;강구수;김서규;이진우;이방원;김윤수;조석팔;하성한
    • Journal of the Korean Institute of Telematics and Electronics B
    • /
    • v.31B no.4
    • /
    • pp.30-39
    • /
    • 1994
  • A 1-chip high quality binarizing VLSI image processor (which has 8 bit ADC. 6 bit flash ADC, 15K standard cell, and 1K word ROM) based on 10 MIPS 16 bit DSP is implemented for FAX. This image processor(IP) performs image pre-processing. image quality improvement in copying and sending mode, and mixed image processing based on the fuzzy theory. And smoothing in sub-scan direction is applied for normal receiving mode data so the received data is enhanced like fine mode data. Each algorithm is processed with the same type of image processing window and 2-D image processing is implemented with a 1-D line buffer. The fabricated chip is applied to a FAX machine and image quality improvement is verified.

  • PDF

Multiple Vehicle Tracking in Urban Environment using Integrated Probabilistic Data Association Filter with Single Laser Scanner (단일 레이저 스캐너와 Integrated Probabilistic Data Association Filter를 이용한 도심환경에서의 다중 차량추적)

  • Kim, Dongchul;Han, Jaehyun;Sunwoo, Myoungho
    • Transactions of the Korean Society of Automotive Engineers
    • /
    • v.21 no.4
    • /
    • pp.33-42
    • /
    • 2013
  • This paper describes a multiple vehicle tracking algorithm using an integrated probabilistic data association filter (IPDAF) in urban environments. The algorithm consists of two parts; a pre-processing stage and an IPDA tracker. In the pre-processing stage, measurements are generated by a feature extraction method that manipulates raw data into predefined geometric features of vehicles as lines and boxes. After that, the measurements are divided into two different objects, dynamic and static objects, by using information of ego-vehicle motion. The IPDA tracker estimates not only states of tracks but also existence probability recursively. The existence probability greatly assists reliable initiation and termination of track in cluttered environment. The algorithm was validated by using experimental data which is collected in urban environment by using single laser scanner.

A Study on Object-Oriented Preprocessing Program for Finite Element Structural Analysis (유한요소 구조해석을 위한 객체지향 전처리 프로그램에 관한 연구)

  • 신영식;서진국;송준엽;우광성
    • Proceedings of the Computational Structural Engineering Institute Conference
    • /
    • 1994.04a
    • /
    • pp.25-32
    • /
    • 1994
  • The pre-processor for finite to element structural analysis considering the user-friendly device is developed by using GUI. This can be used on WINDOWS' environment which is realized the multi-tasking and the concurrency by object-oriented paradigm. Data input can be done easily through menu, dialog box, automatic stepwise input and concurrent representation with the structural geometry on multiple windows. It in designed to control integratedly the pre-processing, execution and the post-processing of the finite element structural analysis program on multiple windows, and input data can be seen with result outputs at the same time. In addition, the object-oriented programming environment makes convenient revision and addition of the program components for expanding the scope of analysis and making better user environment.

  • PDF