• Title/Summary/Keyword: noisy data

Search Result 423, Processing Time 0.023 seconds

REGULARIZED SOLUTION TO THE FREDHOLM INTEGRAL EQUATION OF THE FIRST KIND WITH NOISY DATA

  • Wen, Jin;Wei, Ting
    • Journal of applied mathematics & informatics
    • /
    • v.29 no.1_2
    • /
    • pp.23-37
    • /
    • 2011
  • In this paper, we use a modified Tikhonov regularization method to solve the Fredholm integral equation of the first kind. Under the assumption that measured data are contaminated with deterministic errors, we give two error estimates. The convergence rates can be obtained under the suitable choices of regularization parameters and the number of measured points. Some numerical experiments show that the proposed method is effective and stable.

Improvement of Network Intrusion Detection Rate by Using LBG Algorithm Based Data Mining (LBG 알고리즘 기반 데이터마이닝을 이용한 네트워크 침입 탐지율 향상)

  • Park, Seong-Chul;Kim, Jun-Tae
    • Journal of Intelligence and Information Systems
    • /
    • v.15 no.4
    • /
    • pp.23-36
    • /
    • 2009
  • Network intrusion detection have been continuously improved by using data mining techniques. There are two kinds of methods in intrusion detection using data mining-supervised learning with class label and unsupervised learning without class label. In this paper we have studied the way of improving network intrusion detection accuracy by using LBG clustering algorithm which is one of unsupervised learning methods. The K-means method, that starts with random initial centroids and performs clustering based on the Euclidean distance, is vulnerable to noisy data and outliers. The nonuniform binary split algorithm uses binary decomposition without assigning initial values, and it is relatively fast. In this paper we applied the EM(Expectation Maximization) based LBG algorithm that incorporates the strength of two algorithms to intrusion detection. The experimental results using the KDD cup dataset showed that the accuracy of detection can be improved by using the LBG algorithm.

  • PDF

Research on Deep Learning Performance Improvement for Similar Image Classification (유사 이미지 분류를 위한 딥 러닝 성능 향상 기법 연구)

  • Lim, Dong-Jin;Kim, Taehong
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.8
    • /
    • pp.1-9
    • /
    • 2021
  • Deep learning in computer vision has made accelerated improvement over a short period but large-scale learning data and computing power are still essential that required time-consuming trial and error tasks are involved to derive an optimal network model. In this study, we propose a similar image classification performance improvement method based on CR (Confusion Rate) that considers only the characteristics of the data itself regardless of network optimization or data reinforcement. The proposed method is a technique that improves the performance of the deep learning model by calculating the CRs for images in a dataset with similar characteristics and reflecting it in the weight of the Loss Function. Also, the CR-based recognition method is advantageous for image identification with high similarity because it enables image recognition in consideration of similarity between classes. As a result of applying the proposed method to the Resnet18 model, it showed a performance improvement of 0.22% in HanDB and 3.38% in Animal-10N. The proposed method is expected to be the basis for artificial intelligence research using noisy labeled data accompanying large-scale learning data.

Iterative LBG Clustering for SIMO Channel Identification

  • Daneshgaran, Fred;Laddomada, Massimiliano
    • Journal of Communications and Networks
    • /
    • v.5 no.2
    • /
    • pp.157-166
    • /
    • 2003
  • This paper deals with the problem of channel identification for Single Input Multiple Output (SIMO) slow fading channels using clustering algorithms. Due to the intrinsic memory of the discrete-time model of the channel, over short observation periods, the received data vectors of the SIMO model are spread in clusters because of the AWGN noise. Each cluster is practically centered around the ideal channel output labels without noise and the noisy received vectors are distributed according to a multivariate Gaussian distribution. Starting from the Markov SIMO channel model, simultaneous maximum ikelihood estimation of the input vector and the channel coefficients reduce to one of obtaining the values of this pair that minimizes the sum of the Euclidean norms between the received and the estimated output vectors. Viterbi algorithm can be used for this purpose provided the trellis diagram of the Markov model can be labeled with the noiseless channel outputs. The problem of identification of the ideal channel outputs, which is the focus of this paper, is then equivalent to designing a Vector Quantizer (VQ) from a training set corresponding to the observed noisy channel outputs. The Linde-Buzo-Gray (LBG)-type clustering algorithms [1] could be used to obtain the noiseless channel output labels from the noisy received vectors. One problem with the use of such algorithms for blind time-varying channel identification is the codebook initialization. This paper looks at two critical issues with regards to the use of VQ for channel identification. The first has to deal with the applicability of this technique in general; we present theoretical results for the conditions under which the technique may be applicable. The second aims at overcoming the codebook initialization problem by proposing a novel approach which attempts to make the first phase of the channel estimation faster than the classical codebook initialization methods. Sample simulation results are provided confirming the effectiveness of the proposed initialization technique.

Design of Speech Enhancement U-Net for Embedded Computing (임베디드 연산을 위한 잡음에서 음성추출 U-Net 설계)

  • Kim, Hyun-Don
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.15 no.5
    • /
    • pp.227-234
    • /
    • 2020
  • In this paper, we propose wav-U-Net to improve speech enhancement in heavy noisy environments, and it has implemented three principal techniques. First, as input data, we use 128 modified Mel-scale filter banks which can reduce computational burden instead of 512 frequency bins. Mel-scale aims to mimic the non-linear human ear perception of sound by being more discriminative at lower frequencies and less discriminative at higher frequencies. Therefore, Mel-scale is the suitable feature considering both performance and computing power because our proposed network focuses on speech signals. Second, we add a simple ResNet as pre-processing that helps our proposed network make estimated speech signals clear and suppress high-frequency noises. Finally, the proposed U-Net model shows significant performance regardless of the kinds of noise. Especially, despite using a single channel, we confirmed that it can well deal with non-stationary noises whose frequency properties are dynamically changed, and it is possible to estimate speech signals from noisy speech signals even in extremely noisy environments where noises are much lauder than speech (less than SNR 0dB). The performance on our proposed wav-U-Net was improved by about 200% on SDR and 460% on NSDR compared to the conventional Jansson's wav-U-Net. Also, it was confirmed that the processing time of out wav-U-Net with 128 modified Mel-scale filter banks was about 2.7 times faster than the common wav-U-Net with 512 frequency bins as input values.

A Noise Robust Speech Recognition Method Using Model Compensation Based on Speech Enhancement (음성 개선 기반의 모델 보상 기법을 이용한 강인한 잡음 음성 인식)

  • Shen, Guang-Hu;Jung, Ho-Youl;Chung, Hyun-Yeol
    • The Journal of the Acoustical Society of Korea
    • /
    • v.27 no.4
    • /
    • pp.191-199
    • /
    • 2008
  • In this paper, we propose a MWF-PMC noise processing method which enhances the input speech by using Mel-warped Wiener Filtering (MWF) at pre-processing stage and compensates the recognition model by using PMC (Parallel Model Combination) at post-processing stage for speech recognition in noisy environments. The PMC uses the residual noise extracted from the silence region of enhanced speech at pre-processing stage to compensate the clean speech model and thus this method is considered to improve the performance of speech recognition in noisy environments. For recognition experiments we dew.-sampled KLE PBW (Phoneme Balanced Words) 452 word speech data to 8kHz and made 5 different SNR levels of noisy speech, i.e., 0dB. 5dB, 10dB, 15dB and 20dB, by adding Subway, Car and Exhibition noise to clean speech. From the recognition results, we could confirm the effectiveness of the proposed MWF-PMC method by obtaining the improved recognition performances over all compared with the existing combined methods.

Robust Feature Normalization Scheme Using Separated Eigenspace in Noisy Environments (분리된 고유공간을 이용한 잡음환경에 강인한 특징 정규화 기법)

  • Lee Yoonjae;Ko Hanseok
    • The Journal of the Acoustical Society of Korea
    • /
    • v.24 no.4
    • /
    • pp.210-216
    • /
    • 2005
  • We Propose a new feature normalization scheme based on eigenspace for achieving robust speech recognition. In general, mean and variance normalization (MVN) is Performed in cepstral domain. However, another MVN approach using eigenspace was recently introduced. in that the eigenspace normalization Procedure Performs normalization in a single eigenspace. This Procedure consists of linear PCA matrix feature transformation followed by mean and variance normalization of the transformed cepstral feature. In this method. 39 dimensional feature distribution is represented using only a single eigenspace. However it is observed to be insufficient to represent all data distribution using only a sin91e eigenvector. For more specific representation. we apply unique na independent eigenspaces to cepstra, delta and delta-delta cepstra respectively in this Paper. We also normalize training data in eigenspace and get the model from the normalized training data. Finally. a feature space rotation procedure is introduced to reduce the mismatch of training and test data distribution in noisy condition. As a result, we obtained a substantial recognition improvement over the basic eigenspace normalization.

A Dynamic Backoff Adjustment Method of IEEE 802.15.4 Networks for Real-Time Sporadic Data Transmission (비주기적 실시간 데이터 전송을 위한 IEEE 802.15.4 망의 동적 백오프 조정 기법에 대한 연구)

  • Lee, Jung-Il;Kim, Dong-Sung
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.14 no.3
    • /
    • pp.318-325
    • /
    • 2008
  • In this paper, a dynamic backoff adjustment method of IEEE 802.15.4 is proposed for time-critical sporadic data in a noisy factory environment. For this, a superframe of IEEE 802.15.4 is applied to a real-time mixed data (periodic data, sporadic data, and non real-time message) transmission in factory communication systems. To guarantee a channel access of real-time sporadic(non-periodic) data, a transmission method using the dynamic backoff is applied to wireless control networks. For the real-time property, different initial BE, CW parameters are used for the dynamic backoff adjustment method. The simulat-ion results show an enhancement of the real-time performance of sporadic emergency data. The proposed method provides the channel access of real-time sporadic data efficiently, and guarantee real-time transmission simultaneously within a limite-d timeframe.

Local Map Building Using the information of a Range Finder (영역 검출기 정보를 이용한 지역 지도 작성)

  • Ko, Nak-Yong;Choi, Woong;Choi, Jung-Sang
    • Journal of the Korean Society of Manufacturing Technology Engineers
    • /
    • v.9 no.1
    • /
    • pp.102-110
    • /
    • 2000
  • This paper presents an algorithm of local map building for autonomous robot navigation using LASER range finder information. We develop a model of sensor output for a LASER range finder, and obtain an output data of the LASER range finder for a given environment. From the output data, a local map is obtained through the following procedures: (1) filtering of output data to remove noisy and unnecessary data, (2) comparison of filtered data with the original data to restore useful data, (3) thickening of the map obtained from the restored data, and (4) skeletonizing of the thickened map to get a final local map. Through some simulation studies, a map is obtained from the LASER range finder information for a given indoor environment, and is compared with the environment.

  • PDF

Discrimination model using denoising autoencoder-based majority vote classification for reducing false alarm rate

  • Heonyong Lee;Kyungtak Yu;Shiu Kim
    • Nuclear Engineering and Technology
    • /
    • v.55 no.10
    • /
    • pp.3716-3724
    • /
    • 2023
  • Loose parts monitoring and detecting alarm type in real Nuclear Power Plant have challenges such as background noise, insufficient alarm data, and difficulty of distinction between alarm data that occur during start and stop. Although many signal processing methods and alarm determination algorithms have been developed, it is not easy to determine valid alarm and extract the meaning data from alarm signal including background noise. To address these issues, this paper proposes a denoising autoencoder-based majority vote classification. Training and test data are prepared by acquiring alarm data from real NPP and simulation facility for data augmentation, and noisy data is reproduced by adding Gaussian noise. Using DAEs with 3, 5, 7, and 9 layers, features are extracted for each model and classified into neural networks. Finally, the results obtained from each DAE are classified by majority voting. Also, through comparison with other methods, the accuracy and the false alarm rate are compared, and the excellence of the proposed method is confirmed.