• Title/Summary/Keyword: Data normalization

Search Result 483, Processing Time 0.028 seconds

Forecasting realized volatility using data normalization and recurrent neural network

  • Yoonjoo Lee;Dong Wan Shin;Ji Eun Choi
    • Communications for Statistical Applications and Methods
    • /
    • v.31 no.1
    • /
    • pp.105-127
    • /
    • 2024
  • We propose recurrent neural network (RNN) methods for forecasting realized volatility (RV). The data are RVs of ten major stock price indices, four from the US, and six from the EU. Forecasts are made for relative ratio of adjacent RVs instead of the RV itself in order to avoid the out-of-scale issue. Forecasts of RV ratios distribution are first constructed from which those of RVs are computed which are shown to be better than forecasts constructed directly from RV. The apparent asymmetry of RV ratio is addressed by the Piecewise Min-max (PM) normalization. The serial dependence of the ratio data renders us to consider two architectures, long short-term memory (LSTM) and gated recurrent unit (GRU). The hyperparameters of LSTM and GRU are tuned by the nested cross validation. The RNN forecast with the PM normalization and ratio transformation is shown to outperform other forecasts by other RNN models and by benchmarking models of the AR model, the support vector machine (SVM), the deep neural network (DNN), and the convolutional neural network (CNN).

Harmfulness of Denormalization Adopted for Database for Database Performance Enhancement (데이터베이스 성능향상용 역정규화의 무용성)

  • Rhee Hae Kyung
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.42 no.3 s.303
    • /
    • pp.9-16
    • /
    • 2005
  • For designing the database more efficiently, normailzation can be enforced to minimize the degree of unnecessary data redundancy and contribute to enhance data integrity. However, deep normalization tends to provoke multiple way of schema join, which could then induces response time degradation. To mitigate this sort of side effect that the normalization could brought, a number of field studies we observed adopted the idea of denormalization. To measure whether denormalization contributes to response time improvement, we in this paper developed two different data models about customer service system, one with perfect normalization and the other with denormalization, and evaluated their query response time behaviors. Performance results show that normalization case consistently outperforms denormalization case in terms of response time. This study show that the idea of denormalization, quite rarely contributes to that sort of improvement due ironically to the unnecessary data redundancy.

Print-tip Normalization for DNA Microarray Data (DNA 마이크로어레이 자료의 PRINT-TIP별 표준화(NORMALIZATION) 방법)

  • Yi Sung-Gon;Park Taesung;Kang Sung Hyun;Lee Seung-Yeaun;Lee Yang Sung
    • The Korean Journal of Applied Statistics
    • /
    • v.18 no.1
    • /
    • pp.115-127
    • /
    • 2005
  • DNA microarray experiments allow us to study expression of thousands of genes simultaneously, Normalization is a process for removing noises occurred during the microarray experiment, Print-tip is regarded as one main sources of noises, In this paper, we review normalization methods most commonly used in the microarray experiments, Especially, we investigate the effects of print-tips through simulated data sets.

A Brief Verification Study on the Normalization and Translation Invariant of Measurement Data for Seaport Efficiency : DEA Approach (항만효율성 측정 자료의 정규성과 변환 불변성 검증 소고 : DEA접근)

  • Park, Ro-Kyung;Park, Gil-Young
    • Journal of Korea Port Economic Association
    • /
    • v.23 no.2
    • /
    • pp.109-120
    • /
    • 2007
  • The purpose of this paper is to verify the two problems(normalization for the different inputs and outputs data, translation invariant for the negative data) which will be occurred in measuring the seaport DEA(data envelopment analysis) efficiency. The main result is as follow: Normalization and translation invariant in the BCC model for measuring the seaport efficiency by using 26 Korean seaport data in 1995 with two inputs(berthing capacity, cargo handling capacity) and three outputs(import cargo throughput, export cargo throughput, number of ship calls) was verified. The main policy implication of this paper is that the port management authority should collect the more specific data and publish these data on the inputs and outputs in the seaports with consideration of negative(ex. accident numbers in each seaport) and positive value for analyzing the efficiency by the scholars, because normalization and translation invariant in the data was verified.

  • PDF

Evaluation of Physical Correction in Nuclear Medicine Imaging : Normalization Correction (물리적 보정된 핵의학 영상 평가 : 정규화 보정)

  • Park, Chan Rok;Yoon, Seok Hwan;Lee, Hong Jae;Kim, Jin Eui
    • The Korean Journal of Nuclear Medicine Technology
    • /
    • v.21 no.1
    • /
    • pp.29-33
    • /
    • 2017
  • Purpose In this study, we evaluated image by applying normalization factor during 30 days to the PET images. Materials and Methods Normalization factor was acquired during 30 days. We compared with 30 normalization factors. We selected 3 clinical case (PNS study). We applied for normalization factor to PET raw data and evaluated SUV and count (kBq/ml) by drawing ROI to liver and lesion. Results There is no significant difference normalization factor. SUV and count are not different for PET image according to normalization factor. Conclusion We can get a lot of information doing the quality assurance such as performance of sinogram and detector. That's why we need to do quality assurance daily.

  • PDF

Supervised Rank Normalization for Support Vector Machines (SVM을 위한 교사 랭크 정규화)

  • Lee, Soojong;Heo, Gyeongyong
    • Journal of the Korea Society of Computer and Information
    • /
    • v.18 no.11
    • /
    • pp.31-38
    • /
    • 2013
  • Feature normalization as a pre-processing step has been widely used in classification problems to reduce the effect of different scale in each feature dimension and error as a result. Most of the existing methods, however, assume some distribution function on feature distribution. Even worse, existing methods do not use the labels of data points and, as a result, do not guarantee the optimality of the normalization results in classification. In this paper, proposed is a supervised rank normalization which combines rank normalization and a supervised learning technique. The proposed method does not assume any feature distribution like rank normalization and uses class labels of nearest neighbors in classification to reduce error. SVM, in particular, tries to draw a decision boundary in the middle of class overlapping zone, the reduction of data density in that area helps SVM to find a decision boundary reducing generalized error. All the things mentioned above can be verified through experimental results.

Investigation of Airborne LIDAR Intensity data

  • Chang Hwijeong;Cho Woosug
    • Proceedings of the KSRS Conference
    • /
    • 2004.10a
    • /
    • pp.646-649
    • /
    • 2004
  • LiDAR(Light Detection and Ranging) system can record intensity data as well as range data. Recently, LiDAR intensity data is widely used for landcover classification, ancillary data of feature extraction, vegetation species identification, and so on. Since the intensity return value is associated with several factors, same features is not consistent for same flight or multiple flights. This paper investigated correlation between intensity and range data. Once the effects of range was determined, the single flight line normalization and the multiple flight line normalization was performed by an empirical function that was derived from relationship between range and return intensity

  • PDF

Supervised Rank Normalization with Training Sample Selection (학습 샘플 선택을 이용한 교사 랭크 정규화)

  • Heo, Gyeongyong;Choi, Hun;Youn, Joo-Sang
    • Journal of the Korea Society of Computer and Information
    • /
    • v.20 no.1
    • /
    • pp.21-28
    • /
    • 2015
  • Feature normalization as a pre-processing step has been widely used to reduce the effect of different scale in each feature dimension and error rate in classification. Most of the existing normalization methods, however, do not use the class labels of data points and, as a result, do not guarantee the optimality of normalization in classification aspect. A supervised rank normalization method, combination of rank normalization and supervised learning technique, was proposed and demonstrated better result than others. In this paper, another technique, training sample selection, is introduced in supervised feature normalization to reduce classification error more. Training sample selection is a common technique for increasing classification accuracy by removing noisy samples and can be applied in supervised normalization method. Two sample selection measures based on the classes of neighboring samples and the distance to neighboring samples were proposed and both of them showed better results than previous supervised rank normalization method.

A Robust Watermarking Technique Using Affine Transform and Cross-Reference Points (어파인 변형과 교차참조점을 이용한 강인한 워터마킹 기법)

  • Lee, Hang-Chan
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.56 no.3
    • /
    • pp.615-622
    • /
    • 2007
  • In general, Harris detector is commonly used for finding salient points in watermarking systems using feature points. Harris detector is a kind of combined comer and edge detector which is based on neighboring image data distribution, therefore it has some limitation to find accurate salient points after watermark embedding or any kinds of digital attacks. In this paper, we have used cross reference points which use not data distribution but geometrical structure of a normalized image in order to avoid pointing error caused by the distortion of image data. After normalization, we find cross reference points and take inverse normalization of these points. Next, we construct a group of triangles using tessellation with inversely normalized cross reference points. The watermarks are affine transformed and transformed-watermarks are embedded into not normalized image but original one. Only locations of watermarks are determined on the normalized image. Therefore, we can reduce data loss of watermark which is caused by inverse normalization. As a result, we can detect watermarks with high correlation after several digital attacks.

Normalization Factor for Three-Level Hierarchical 64QAM Scheme (3-level 계층 64QAM 기법의 정규화 인수)

  • You, Dongho;Kim, Dong Ho
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.41 no.1
    • /
    • pp.77-79
    • /
    • 2016
  • In this paper, we consider hierarchical modulation (HM), which has been widely exploited in digital broadcasting systems. In HM, each independent data stream is mapped to the modulation symbol with different transmission power and normalization factors of conventional M-QAM cannot be used. In this paper, we derive the method and formula for exact normalization factor of three-level hierarchical 64QAM.