• 제목/요약/키워드: Noisy data reduction

검색결과 31건 처리시간 0.019초

Issues and Empirical Results for Improving Text Classification

  • Ko, Young-Joong;Seo, Jung-Yun
    • Journal of Computing Science and Engineering
    • /
    • 제5권2호
    • /
    • pp.150-160
    • /
    • 2011
  • Automatic text classification has a long history and many studies have been conducted in this field. In particular, many machine learning algorithms and information retrieval techniques have been applied to text classification tasks. Even though much technical progress has been made in text classification, there is still room for improvement in text classification. In this paper, we will discuss remaining issues in improving text classification. In this paper, three improvement issues are presented including automatic training data generation, noisy data treatment and term weighting and indexing, and four actual studies and their empirical results for those issues are introduced. First, the semi-supervised learning technique is applied to text classification to efficiently create training data. For effective noisy data treatment, a noisy data reduction method and a robust text classifier from noisy data are developed as a solution. Finally, the term weighting and indexing technique is revised by reflecting the importance of sentences into term weight calculation using summarization techniques.

An adaptive nonlocal filtering for low-dose CT in both image and projection domains

  • Wang, Yingmei;Fu, Shujun;Li, Wanlong;Zhang, Caiming
    • Journal of Computational Design and Engineering
    • /
    • 제2권2호
    • /
    • pp.113-118
    • /
    • 2015
  • An important problem in low-dose CT is the image quality degradation caused by photon starvation. There are a lot of algorithms in sinogram domain or image domain to solve this problem. In view of strong self-similarity contained in the special sinusoid-like strip data in the sinogram space, we propose a novel non-local filtering, whose average weights are related to both the image FBP (filtered backprojection) reconstructed from restored sinogram data and the image directly FBP reconstructed from noisy sinogram data. In the process of sinogram restoration, we apply a non-local method with smoothness parameters adjusted adaptively to the variance of noisy sinogram data, which makes the method much effective for noise reduction in sinogram domain. Simulation experiments show that our proposed method by filtering in both image and projection domains has a better performance in noise reduction and details preservation in reconstructed images.

적응예측기를 이용하여 잡음섞인 음성신호로부터 autoregressive 계수를 추산하는 방법 (An Autoregressive Parameter Estimation from Noisy Speech Using the Adaptive Predictor)

  • 구본응
    • 한국음향학회지
    • /
    • 제14권3호
    • /
    • pp.90-96
    • /
    • 1995
  • 잡음섞인 관측데이타로부터 AR 모수를 추정하는 방법을 제안하였다. AP 방법이라고 이름붙인 이 방법은 단순하고도 신뢰성있는 적응예측기를 이용하려는 시도의 산물이다. 잡음섞인 입력수열로부터 계산된 AR 모수의 추정치보다 예측수열로부터 계산된 AR 모수의 추정치가 원래의 모수에 스펙트럼상의 거리가 더 가깝다는 것을 이론적으로 증명하였다. 실제 음성 신호와 칼만필터를 사용한 실험결과도 이론과 일치함을 보였다. 대략적으로, AP방법으로 계산된 추정치를 사용하였을때의 잡음감쇠성능은 잡음섞인 입력수열로부터 계산된 AP 모수의 추정치를 사용하였을때보다는 우수하였고, EM반복법에 의한 추정치를 사용하였을때보다는 약간 못한 것으로 나타났다. 그러나, 제안된 방법은 그 단순성으로 인하여 경우에 따라 더 복잡한 다른 방법의 대안으로 사용될 수 있을 것이다.

  • PDF

Eigenvoice를 이용한 이진 마스크 분류 모델 적응 방법 (Eigenvoice Adaptation of Classification Model for Binary Mask Estimation)

  • 김기백
    • 방송공학회논문지
    • /
    • 제20권1호
    • /
    • pp.164-170
    • /
    • 2015
  • 본 논문에서는 잡음 환경에서 취득된 음성 신호에서 잡음을 제거하기 위한 방법으로 사용되는 이진 마스크 분류 모델의 적응과정에 대해 다루고자 한다. 기존 연구결과에 의하면, 잡음 환경 데이터에 이진 마스크 기법을 적용하면 음성 명료도를 향상시킬 수 있다고 알려져 있다. 하지만 이진 마스크 분류 모델 학습 시 테스트 환경 데이터가 포함되어야 한다는 단점을 안고 있다. 본 논문에서는 새로운 잡음 환경에서 이진 마스크 분류 모델을 적응하기 위해, 음성 인식에서 널리 사용되는 화자 적응 기법인 eigenvoice 방법을 적용하고자 한다. 실험결과에서는 모델 적응에 사용되는 데이터량에 따른 성능을 정검출율과 오검출율 관점에서 평가하였고, 그 결과 새로운 잡음 환경에서 데이터량을 증가시켜 모델을 적응함으로써 향상된 성능을 나타냄을 확인할 수 있었다.

위너필터에 의한 음성 중의 잡음제거 알고리즘 (Noise Reduction Algorithm in Speech by Wiener Filter)

  • 최재승
    • 한국전자통신학회논문지
    • /
    • 제8권9호
    • /
    • pp.1293-1298
    • /
    • 2013
  • 본 논문에서는 음성신호를 개선할 목적으로 잡음으로 오염된 음성신호로부터 잡음성분을 제거하기 위한 위너 필터를 사용한 잡음제거 알고리즘을 제안한다. 제안한 알고리즘은 먼저 잡음 복원 및 제거 방법에 기초하여 잡음으로 오염된 신호로부터 각 프레임에서 백색잡음의 잡음 스펙트럼을 제거한다. 또한 본 알고리즘은 선형예측 분석 방법에 기초한 위너 필터를 사용하여 음성신호를 강조한다. 본 실험에서는 일본 남성화자에 의한 음성과 잡음데이터를 사용하여 본 알고리즘의 실험 결과를 나타낸다. 백색잡음에 의하여 오염된 음성신호에 대하여 스펙트럼 왜곡률 척도를 사용하여 본 알고리즘이 유효하다는 것을 확인한다. 실험으로부터 백색잡음에 대하여 이전의 위너 필터와 비교하여 최대 4.94 dB의 출력 스펙트럼 왜곡률이 개선된 것을 확인할 수 있었다.

음성구간 검출기의 실시간 적응화를 위한 음성 특징벡터의 차원 축소 방법 (Dimension Reduction Method of Speech Feature Vector for Real-Time Adaptation of Voice Activity Detection)

  • 박진영;이광석;허강인
    • 융합신호처리학회논문지
    • /
    • 제7권3호
    • /
    • pp.116-121
    • /
    • 2006
  • 본 논문에서는 다양한 잡음환경에서의 실시간 적응화 기법을 적용하기 위한 선결 과제로 다차원 음성 특정 벡터를 저차원으로 축소하는 방법을 제안한다. 제안된 방법은 특징 벡터를 확률 우도 값으로 매핑시켜 비선형적으로 축소하는 방법으로 음성 / 비음성의 분류는 우도비 검증 (Likelihood Ratio Test; LRT) 을 이용하여 분류하였다. 실험 결과 고차원 특징 벡터를 이용하여 분류한 결과와 대등하게 분류됨을 확인할 수 있었다. 그리고, 제안된 방법에 의해 검출된 음성 데이터를 이용한 음성인식 실험에서도 10차 MFCC(Mel-Frequency Cepstral Coefficient)를 사용하여 분류한 경우와 대등한 인식률을 보여주었다.

  • PDF

MR 방법으로부터 다단 정현파의 주파수 추정 (Frequency Estimation of Multiple Sinusoids From MR Method)

  • 안태천;탁현수;이종범
    • 전자공학회논문지B
    • /
    • 제29B권2호
    • /
    • pp.18-26
    • /
    • 1992
  • MR(Model Reduction) is presented in order to estimate the frequency of multiple sinusoids from the finite noisy data with the white or colored noises. MR, using the reduced rank models, is designed, appling the approximation of linear system to LP(Linear Prediction). The MR method is analyzed. Monte-carlo simulations are conducted for MR and Lp. The results are compared with in terms of mean, root-mean square and relative bias. MR eliminates effectevely the extremeous and exceptional poles appearing in LP and improves the accuracy of LP. Especially, MR gives promising results in short noisy measurements, low SNR's and colored noises. Power spectral density and angular frequency position are showed by figures, for examples. Finally, the new method is utilized to the communication and biomedical systems estimating the characteristics of the signal and the system identification modelling the dynamic systems from experimental data.

  • PDF

A Study on Data Classification of Raman OIM Hyperspectral Bone Data

  • Jung, Sung-Hwan
    • 한국멀티미디어학회논문지
    • /
    • 제14권8호
    • /
    • pp.1010-1019
    • /
    • 2011
  • This was a preliminary research for the goal of understanding between internal structure of Osteogenesis Imperfecta Murine (OIM) bone and its fragility. 54 hyperspectral bone data sets were captured by using JASCO 2000 Raman spectrometer at UMKC-CRISP (University of Missouri-Kansas City Center for Research on Interfacial Structure and Properties). Each data set consists of 1,091 data points from 9 OIM bones. The original captured hyperspectral data sets were noisy and base-lined ones. We removed the noise and corrected the base-lined data for the final efficient classification. High dimensional Raman hyperspectral data on OIM bones was reduced by Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) and efficiently classified for the first time. We confirmed OIM bones could be classified such as strong, middle and weak one by using the coefficients of their PCA or LDA. Through experiment, we investigated the efficiency of classification on the reduced OIM bone data by the Bayesian classifier and K -Nearest Neighbor (K-NN) classifier. As the experimental result, the case of LDA reduction showed higher classification performance than that of PCA reduction in the two classifiers. K-NN classifier represented better classification rate, compared with Bayesian classifier. The classification performance of K-NN was about 92.6% in case of LDA.

A NEW METHOD FOR NORTH-SOUTH ASYMMETRY OF SUN SPOT AREA ANALYSIS

  • Chang, Heon-Young
    • Journal of Astronomy and Space Sciences
    • /
    • 제24권4호
    • /
    • pp.261-268
    • /
    • 2007
  • We have studied the temporal variation in the North-South asymmetry of the sunspot area during the period from 1874 to 2007. Though the 9-year periodicity is commonly reported, shorter periodicities is still under study. We employ the cepstrum analysis method to analyze the noisy power spectrum of the North-South asymmetry. We demonstrate that the cleaned power spectrum shows reduction of the spurious back-ground noise level. Some of short period peaks in the power spectrum disappear after deconvolution. It should be, however, pointed out that power spectrum might look less noisy because of a filtering process during deconvolution. We conclude by pointing out that a more sophisticate filtering algorithm is required to produce a precise and reliable periodicity estimate.

EER-ASSL: Combining Rollback Learning and Deep Learning for Rapid Adaptive Object Detection

  • Ahmed, Minhaz Uddin;Kim, Yeong Hyeon;Rhee, Phill Kyu
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제14권12호
    • /
    • pp.4776-4794
    • /
    • 2020
  • We propose a rapid adaptive learning framework for streaming object detection, called EER-ASSL. The method combines the expected error reduction (EER) dependent rollback learning and the active semi-supervised learning (ASSL) for a rapid adaptive CNN detector. Most CNN object detectors are built on the assumption of static data distribution. However, images are often noisy and biased, and the data distribution is imbalanced in a real world environment. The proposed method consists of collaborative sampling and EER-ASSL. The EER-ASSL utilizes the active learning (AL) and rollback based semi-supervised learning (SSL). The AL allows us to select more informative and representative samples measuring uncertainty and diversity. The SSL divides the selected streaming image samples into the bins and each bin repeatedly transfers the discriminative knowledge of the EER and CNN models to the next bin until convergence and incorporation with the EER rollback learning algorithm is achieved. The EER models provide a rapid short-term myopic adaptation and the CNN models an incremental long-term performance improvement. EER-ASSL can overcome noisy and biased labels in varying data distribution. Extensive experiments shows that EER-ASSL obtained 70.9 mAP compared to state-of-the-art technology such as Faster RCNN, SSD300, and YOLOv2.