• 제목/요약/키워드: data pre-processing

검색결과 805건 처리시간 0.024초

An Improved Detection Technique for Spread Spectrum Audio Watermarking with a Spectral Envelope Filter

  • Jung, Sa-Rah;Seok, Jong-Won;Hong, Jin-Woo
    • ETRI Journal
    • /
    • 제25권1호
    • /
    • pp.52-54
    • /
    • 2003
  • We propose an improved algorithm for detecting audio watermarks based on a spread spectrum in the spectral domain. Since the energy of a watermark is much smaller than that of the cover audio data, pre-processing to reduce the effect of the cover data is needed to reliably extract watermarks. We introduce a spectral envelope filter as a pre-process that enhances detecting performance by filtering out the intrinsic spectral character of cover data. The proposed watermarking structure can be easily included in the compression system and can extract watermarks from partially decompressed spectral data. Our experimental results demonstrate that with a bit error rate of around 10 dB against general attacks, the proposed detecting scheme works better than detectors without the spectral filter.

  • PDF

PAIVS: prediction of avian influenza virus subtype

  • Park, Hyeon-Chun;Shin, Juyoun;Cho, Sung-Min;Kang, Shinseok;Chung, Yeun-Jun;Jung, Seung-Hyun
    • Genomics & Informatics
    • /
    • 제18권1호
    • /
    • pp.5.1-5.5
    • /
    • 2020
  • Highly pathogenic avian influenza (HPAI) viruses have caused severe respiratory disease and death in poultry and human beings. Although most of the avian influenza viruses (AIVs) are of low pathogenicity and cause mild infections in birds, some subtypes including hemagglutinin H5 and H7 subtype cause HPAI. Therefore, sensitive and accurate subtyping of AIV is important to prepare and prevent for the spread of HPAI. Next-generation sequencing (NGS) can analyze the full-length sequence information of entire AIV genome at once, so this technology is becoming a more common in detecting AIVs and predicting subtypes. However, an analysis pipeline of NGS-based AIV sequencing data, including AIV subtyping, has not yet been established. Here, in order to support the pre-processing of NGS data and its interpretation, we developed a user-friendly tool, named prediction of avian influenza virus subtype (PAIVS). PAIVS has multiple functions that support the pre-processing of NGS data, reference-guided AIV subtyping, de novo assembly, variant calling and identifying the closest full-length sequences by BLAST, and provide the graphical summary to the end users.

Application of recursive SSA as data pre-processing filter for stochastic subspace identification

  • Loh, Chin-Hsiung;Liu, Yi-Cheng
    • Smart Structures and Systems
    • /
    • 제11권1호
    • /
    • pp.19-34
    • /
    • 2013
  • The objective of this paper is to develop on-line system parameter estimation and damage detection technique from the response measurements through using the Recursive Covariance-Driven Stochastic Subspace identification (RSSI-COV) approach. To reduce the effect of noise on the results of identification, discussion on the pre-processing of data using recursive singular spectrum analysis (rSSA) is presented to remove the noise contaminant measurements so as to enhance the stability of data analysis. Through the application of rSSA-SSI-COV to the vibration measurement of bridge during scouring experiment, the ability of the proposed algorithm was proved to be robust to the noise perturbations and offers a very good online tracking capability. The accuracy and robustness offered by rSSA-SSI-COV provides a key to obtain the evidence of imminent bridge settlement and a very stable modal frequency tracking which makes it possible for early warning. The peak values of the identified $1^{st}$ mode shape slope ratio has shown to be a good indicator for damage location, meanwhile, the drastic movements of the peak of $2^{nd}$ mode slope ratio could be used as another feature to indicate imminent pier settlement.

Preprocessing of the Direct-broadcast Data from the Atmospheric Infared Sounder (AIRS) Sounding Suite on Aqua Satellite

  • 김성범;박혜숙;김금란;박승환;김문규;이종규
    • 대기
    • /
    • 제13권4호
    • /
    • pp.71-79
    • /
    • 2003
  • We present a pre processing system for the Atmospheric Infrared Sounder (AIRS) sounding suite onboard Aqua satellite. With its unprecedented 2378 channels in IR bands, AIRS aims at achieving the sounding accuracy [s1]of a radiosonde (1 K in 1-km layer for temperature and 10% in 2-km layer for humidity). The core of the pre p rocessor is the International MODIS/AIRS Processing Package (IMAPP) that performs the geometric and radiometric correction to compute the Earth's radiance. Then we remove spurious data and retrieve the brightness temperature (Tb). Since we process the direct-broadcast data almost for the first time among the AIRS directbroadcast community, special attention is needed to understand and verify the products. This includes the pixel-to-pixel verification of the direct-broadcast product with reference to the fullorbit product, which shows the difference of less than $10^{-3}$ K in IR Tb.

Crop Leaf Disease Identification Using Deep Transfer Learning

  • Changjian Zhou;Yutong Zhang;Wenzhong Zhao
    • Journal of Information Processing Systems
    • /
    • 제20권2호
    • /
    • pp.149-158
    • /
    • 2024
  • Traditional manual identification of crop leaf diseases is challenging. Owing to the limitations in manpower and resources, it is challenging to explore crop diseases on a large scale. The emergence of artificial intelligence technologies, particularly the extensive application of deep learning technologies, is expected to overcome these challenges and greatly improve the accuracy and efficiency of crop disease identification. Crop leaf disease identification models have been designed and trained using large-scale training data, enabling them to predict different categories of diseases from unlabeled crop leaves. However, these models, which possess strong feature representation capabilities, require substantial training data, and there is often a shortage of such datasets in practical farming scenarios. To address this issue and improve the feature learning abilities of models, this study proposes a deep transfer learning adaptation strategy. The novel proposed method aims to transfer the weights and parameters from pre-trained models in similar large-scale training datasets, such as ImageNet. ImageNet pre-trained weights are adopted and fine-tuned with the features of crop leaf diseases to improve prediction ability. In this study, we collected 16,060 crop leaf disease images, spanning 12 categories, for training. The experimental results demonstrate that an impressive accuracy of 98% is achieved using the proposed method on the transferred ResNet-50 model, thereby confirming the effectiveness of our transfer learning approach.

A CPU-GPU Hybrid System of Environment Perception and 3D Terrain Reconstruction for Unmanned Ground Vehicle

  • Song, Wei;Zou, Shuanghui;Tian, Yifei;Sun, Su;Fong, Simon;Cho, Kyungeun;Qiu, Lvyang
    • Journal of Information Processing Systems
    • /
    • 제14권6호
    • /
    • pp.1445-1456
    • /
    • 2018
  • Environment perception and three-dimensional (3D) reconstruction tasks are used to provide unmanned ground vehicle (UGV) with driving awareness interfaces. The speed of obstacle segmentation and surrounding terrain reconstruction crucially influences decision making in UGVs. To increase the processing speed of environment information analysis, we develop a CPU-GPU hybrid system of automatic environment perception and 3D terrain reconstruction based on the integration of multiple sensors. The system consists of three functional modules, namely, multi-sensor data collection and pre-processing, environment perception, and 3D reconstruction. To integrate individual datasets collected from different sensors, the pre-processing function registers the sensed LiDAR (light detection and ranging) point clouds, video sequences, and motion information into a global terrain model after filtering redundant and noise data according to the redundancy removal principle. In the environment perception module, the registered discrete points are clustered into ground surface and individual objects by using a ground segmentation method and a connected component labeling algorithm. The estimated ground surface and non-ground objects indicate the terrain to be traversed and obstacles in the environment, thus creating driving awareness. The 3D reconstruction module calibrates the projection matrix between the mounted LiDAR and cameras to map the local point clouds onto the captured video images. Texture meshes and color particle models are used to reconstruct the ground surface and objects of the 3D terrain model, respectively. To accelerate the proposed system, we apply the GPU parallel computation method to implement the applied computer graphics and image processing algorithms in parallel.

Number of sampling leaves for reflectance measurement of Chinese cabbage and kale

  • Chung, Sun-Ok;Ngo, Viet-Duc;Kabir, Md. Shaha Nur;Hong, Soon-Jung;Park, Sang-Un;Kim, Sun-Ju;Park, Jong-Tae
    • 농업과학연구
    • /
    • 제41권3호
    • /
    • pp.169-175
    • /
    • 2014
  • Objective of this study was to investigate effects of pre-processing method and number of sampling leaves on stability of the reflectance measurement for Chinese cabbage and kale leaves. Chinese cabbage and kale were transplanted and cultivated in a plant factory. Leaf samples of the kale and cabbage were collected at 4 weeks after transplanting of the seedlings. Spectra data were collected with an UV/VIS/NIR spectrometer in the wavelength region from 190 to 1130 nm. All leaves (mature and young leaves) were measured on 9 and 12 points in the blade part in the upper area for kale and cabbage leaves, respectively. To reduce the spectral noise, the raw spectral data were preprocessed by different methods: i) moving average, ii) Savitzky-Golay filter, iii) local regression using weighted linear least squares and a $1^{st}$ degree polynomial model (lowess), iv) local regression using weighted linear least squares and a $2^{nd}$ degree polynomial model (loess), v) a robust version of 'lowess', vi) a robust version of 'loess', with 7, 11, 15 smoothing points. Effects of number of sampling leaves were investigated by reflectance difference (RD) and cross-correlation (CC) methods. Results indicated that the contribution of the spectral data collected at 4 sampling leaves were good for both of the crops for reflectance measurement that does not change stability of measurement much. Furthermore, moving average method with 11 smoothing points was believed to provide reliable pre-processed data for further analysis.

프로세서 노드 상황을 고려하는 효율적인 메시지 스캐터 및 개더 알고리즘 (Efficient Message Scattering and Gathering Based on Processing Node Status)

  • Park, Jongsu
    • 한국정보통신학회논문지
    • /
    • 제26권4호
    • /
    • pp.637-640
    • /
    • 2022
  • To maximize performance in a high-performance multicore processor system. it is essential to enable effective data communication between processing cores. Data communication between processor nodes can be broadly classified into collective and point-to-point communications. Collective communication comprises scattering and gathering. This paper presents a efficient message scattering and gathering based on processing node status. In the proposed algorithms, the transmission order is changed according to the data size of the pre-existing communication, to reduce the waiting time required until the collective communications begin. From the simulation, the performances of the proposed message scattering and gathering algorithms were improved by approximately 71.41% and 69.84%.

LDA기반 토픽모델링을 활용한 공공데이터 기반의 교육용 데이터마이닝 연구 (A Study on Educational Data Mining for Public Data Portal through Topic Modeling Method with Latent Dirichlet Allocation)

  • 신승기
    • 정보교육학회논문지
    • /
    • 제26권5호
    • /
    • pp.439-448
    • /
    • 2022
  • 본 연구에서는 공공데이터포털에서 제공하는 교육관련 데이터를 검색하고 토픽모델링 기법을 활용한 분류를 통해 어떠한 데이터의 종류가 구축되어 있으며 활용이 가능한지를 살펴보고자 하였다. 공공데이터포털의 데이터에 대하여 분류체계를 기준으로 교육분야의 파일데이터는 3,072건이 수집되었으며, 검색어를 활용하여 '교육'을 검색하여 나타난 파일데이터 2,361건으로 나타났다. 각각의 데이터셋에 대하여 불용어처리를 실시하고 데이터 전처리를 수행하여 LDA기반 토픽모델링을 활용하여 텍스트마이닝 분석을 실시하였다. 사전에 교육으로 분류된 데이터셋에서는 현재 재학중인 학교급별 학생을 대상으로 지원하는 프로그램과 정보에 대한 내용이 제공되고 있었다. 한편, 교육으로 검색하여 수집된 데이터셋에서는 장애인, 학부모, 노인, 아동 등 평생교육의 관점으로 제공되는 교육 프로그램 및 지원현황이라는 특징이 나타났다. 데이터과학기반의 의사결정 및 문제해결력을 기르기 위해 공공데이터포털이 제공하는 데이터에서 교육과정 및 내용이 충분히 제공되는 것도 좋은 기회가 될 것이다.

도시하천관리 연계 플랫폼 개발(I) (Development of urban river data management platform(I))

  • 이성학;심규철;구본현
    • 한국수자원학회논문집
    • /
    • 제52권12호
    • /
    • pp.1087-1098
    • /
    • 2019
  • 본 연구에서는 도시하천관리에 있어 이용되는 데이터를 수집, 정제 및 제공하는 기능을 수행하는 도시하천 통합데이터 플랫폼 개발을 수행하였다. 도시하천 통합데이터 플랫폼은 Open API서비스를 활용하여 다양한 기관에서 제공되는 데이터를 수집하는 기능을 가지고 있으며, 수집된 데이터는 전처리 과정을 통하여 정제되어 데이터베이스에 적재된다. 수집된 데이터는 시각화 시스템을 활용하여 검토 및 분석이 가능하며, 단위 Open API 형태로 제공되므로 도시하천모형에서 이를 조합하여 개별적인 입력자료로 활용할 수 있도록 하였다. 또한 실시간 데이터에 대한 제공시스템을 개발하여 도시하천모형에 실시간 데이터를 적용할 수 있도록 하였다. 이를 통하여 사용자는 데이터의 수집과 전처리, 입력자료 구축에 필요한 시간과 노력을 절감하여 도시하천관리 모형과 시스템의 개발에 있어 효율성과 확장성이 증대될 것으로 판단된다.