• Title/Summary/Keyword: Sparse data

Search Result 408, Processing Time 0.022 seconds

Image Denoising Using Nonlocal Similarity and 3D Filtering (비지역적 유사성 및 3차원 필터링 기반 영상 잡음제거)

  • Kim, Seehyun
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.10
    • /
    • pp.1886-1891
    • /
    • 2017
  • Denoising which is one of major research topics in the image processing deals with recovering the noisy images. Natural images are well known not only for their local but also nonlocal similarity. Patterns of unique edges and texture which are crucial for understanding the image are repeated over the nonlocal region. In this paper, a nonlocal similarity based denoising algorithm is proposed. First for every blocks of the noisy image, nonlocal similar blocks are gathered to construct a overcomplete data set which are inherently sparse in the transform domain due to the characteristics of the images. Then, the sparse transform coefficients are filtered to suppress the non-sparse additive noise. Finally, the image is recovered by aggregating the overcomplete estimates of each pixel. Performance experiments with several images show that the proposed algorithm outperforms the conventional methods in removing the additive Gaussian noise effectively while preserving the image details.

Application of couple sparse coding ensemble on structural damage detection

  • Fallahian, Milad;Khoshnoudian, Faramarz;Talaei, Saeid
    • Smart Structures and Systems
    • /
    • v.21 no.1
    • /
    • pp.1-14
    • /
    • 2018
  • A method is proposed to detect structural damages in the presence of damping using noisy data. This method uses Frequency Response Function (FRF) and Mode-Shapes as the input parameters for a system of Couple Sparse Coding (CSC) to study the healthy state of the structure. To obtain appropriate patterns of FRF for CSC training, Principal Component Analysis (PCA) technique is adopted to reduce the full-size FRF to overcome over-fitting and convergence problems in machine-learning training. To verify the proposed method, a numerical two-story frame structure is employed. A system of individual CSCs is trained with FRFs and mode-shapes, and then termed ensemble to detect the health condition of the structure. The results demonstrate that the proposed method is accurate in damage identification even in presence of up to 20% noisy data and 5% unconsidered damping ratio. Furthermore, it can be concluded that CSC ensemble is highly efficient to detect the location and the severity of damages in comparison to the individual CSC trained only with FRF data.

GOODNESS-OF-FIT TEST USING LOCAL MAXIMUM LIKELIHOOD POLYNOMIAL ESTIMATOR FOR SPARSE MULTINOMIAL DATA

  • Baek, Jang-Sun
    • Journal of the Korean Statistical Society
    • /
    • v.33 no.3
    • /
    • pp.313-321
    • /
    • 2004
  • We consider the problem of testing cell probabilities in sparse multinomial data. Aerts et al. (2000) presented T=${{\Sigma}_{i=1}}^{k}{[{p_i}^{*}-E{(p_{i}}^{*})]^2$ as a test statistic with the local least square polynomial estimator ${{p}_{i}}^{*}$, and derived its asymptotic distribution. The local least square estimator may produce negative estimates for cell probabilities. The local maximum likelihood polynomial estimator ${{\hat{p}}_{i}}$, however, guarantees positive estimates for cell probabilities and has the same asymptotic performance as the local least square estimator (Baek and Park, 2003). When there are cell probabilities with relatively much different sizes, the same contribution of the difference between the estimator and the hypothetical probability at each cell in their test statistic would not be proper to measure the total goodness-of-fit. We consider a Pearson type of goodness-of-fit test statistic, $T_1={{\Sigma}_{i=1}}^{k}{[{p_i}^{*}-E{(p_{i}}^{*})]^2/p_{i}$ instead, and show it follows an asymptotic normal distribution. Also we investigate the asymptotic normality of $T_2={{\Sigma}_{i=1}}^{k}{[{p_i}^{*}-E{(p_{i}}^{*})]^2/p_{i}$ where the minimum expected cell frequency is very small.

Dense Sub-Cube Extraction Algorithm for a Multidimensional Large Sparse Data Cube (다차원 대용량 저밀도 데이타 큐브에 대한 고밀도 서브 큐브 추출 알고리즘)

  • Lee Seok-Lyong;Chun Seok-Ju;Chung Chin-Wan
    • Journal of KIISE:Databases
    • /
    • v.33 no.4
    • /
    • pp.353-362
    • /
    • 2006
  • A data warehouse is a data repository that enables users to store large volume of data and to analyze it effectively. In this research, we investigate an algorithm to establish a multidimensional data cube which is a powerful analysis tool for the contents of data warehouses and databases. There exists an inevitable retrieval overhead in a multidimensional data cube due to the sparsity of the cube. In this paper, we propose a dense sub-cube extraction algorithm that identifies dense regions from a large sparse data cube and constructs the sub-cubes based on the dense regions found. It reduces the retrieval overhead remarkably by retrieving those small dense sub-cubes instead of scanning a large sparse cube. The algorithm utilizes the bitmap and histogram based techniques to extract dense sub-cubes from the data cube, and its effectiveness is demonstrated via an experiment.

An Improved RSR Method to Obtain the Sparse Projection Matrix (희소 투영행렬 획득을 위한 RSR 개선 방법론)

  • Ahn, Jung-Ho
    • Journal of Digital Contents Society
    • /
    • v.16 no.4
    • /
    • pp.605-613
    • /
    • 2015
  • This paper addresses the problem to make sparse the projection matrix in pattern recognition method. Recently, the size of computer program is often restricted in embedded systems. It is very often that developed programs include some constant data. For example, many pattern recognition programs use the projection matrix for dimension reduction. To improve the recognition performance, very high dimensional feature vectors are often extracted. In this case, the projection matrix can be very big. Recently, RSR(roated sparse regression) method[1] was proposed. This method has been proved one of the best algorithm that obtains the sparse matrix. We propose three methods to improve the RSR; outlier removal, sampling and elastic net RSR(E-RSR) in which the penalty term in RSR optimization function is replaced by that of the elastic net regression. The experimental results show that the proposed methods are very effective and improve the sparsity rate dramatically without sacrificing the recognition rate compared to the original RSR method.

Design of FIR Filters With Sparse Signed Digit Coefficients (희소한 부호 자리수 계수를 갖는 FIR 필터 설계)

  • Kim, Seehyun
    • Journal of IKEEE
    • /
    • v.19 no.3
    • /
    • pp.342-348
    • /
    • 2015
  • High speed implementation of digital filters is required in high data rate applications such as hard-wired wide band modem and high resolution video codec. Since the critical path of the digital filter is the MAC (multiplication and accumulation) circuit, the filter coefficient with sparse non-zero bits enables high speed implementation with adders of low hardware cost. Compressive sensing has been reported to be very successful in sparse representation and sparse signal recovery. In this paper a filter design method for digital FIR filters with CSD (canonic signed digit) coefficients using compressive sensing technique is proposed. The sparse non-zero signed bits are selected in the greedy fashion while pruning the mistakenly selected digits. A few design examples show that the proposed method can be utilized for designing sparse CSD coefficient digital FIR filters approximating the desired frequency response.

Face Recognition via Sparse Representation using the ROMP Method (ROMP를 이용한 희소 표현 방식 얼굴 인식 방법론)

  • Ahn, Jung-Ho;Choi, KwonTaeg
    • Journal of Digital Contents Society
    • /
    • v.18 no.2
    • /
    • pp.347-356
    • /
    • 2017
  • It is well-known that the face recognition method via sparse representation has been proved very robust and showed good performance. Its weakness is, however, that its time complexity is very high because it should solve $L_1$-minimization problem to find the sparse solution. In this paper, we propose to use the ROMP(Regularized Orthogonal Matching Pursuit) method for the sparse solution, which solves the $L_2$-minimization problem with regularization condition using the greed strategy. In experiments, we shows that the proposed method is comparable to the existing best $L_1$-minimization solver, Homotopy, but is 60 times faster than Homotopy. Also, we proposed C-SCI method for classification. The C-SCI method is very effective since it considers the sparse solution only without reconstructing the test data. It is shown that the C-SCI method is comparable to, but is 5 times faster than the existing best classification method.

Sparse reconstruction of guided wavefield from limited measurements using compressed sensing

  • Qiao, Baijie;Mao, Zhu;Sun, Hao;Chen, Songmao;Chen, Xuefeng
    • Smart Structures and Systems
    • /
    • v.25 no.3
    • /
    • pp.369-384
    • /
    • 2020
  • A wavefield sparse reconstruction technique based on compressed sensing is developed in this work to dramatically reduce the number of measurements. Firstly, a severely underdetermined representation of guided wavefield at a snapshot is established in the spatial domain. Secondly, an optimal compressed sensing model of guided wavefield sparse reconstruction is established based on l1-norm penalty, where a suite of discrete cosine functions is selected as the dictionary to promote the sparsity. The regular, random and jittered undersampling schemes are compared and selected as the undersampling matrix of compressed sensing. Thirdly, a gradient projection method is employed to solve the compressed sensing model of wavefield sparse reconstruction from highly incomplete measurements. Finally, experiments with different excitation frequencies are conducted on an aluminum plate to verify the effectiveness of the proposed sparse reconstruction method, where a scanning laser Doppler vibrometer as the true benchmark is used to measure the original wavefield in a given inspection region. Experiments demonstrate that the missing wavefield data can be accurately reconstructed from less than 12% of the original measurements; The reconstruction accuracy of the jittered undersampling scheme is slightly higher than that of the random undersampling scheme in high probability, but the regular undersampling scheme fails to reconstruct the wavefield image; A quantified mapping relationship between the sparsity ratio and the recovery error over a special interval is established with respect to statistical modeling and analysis.

A Sparse-ON Pixel Two-Dimensional 4-Level 4/6 Balanced-Modulation Code in Holographic Data Storage Systems (홀로그래픽 데이터 저장장치를 위한 저밀도 ON 픽셀 2차원 4-레벨 4/6 균형 변조부호)

  • Park, Keunhwan;Lee, Jaejin
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.53 no.11
    • /
    • pp.9-14
    • /
    • 2016
  • In the holographic data storage system, the data can be stored more than one bit per pixel and the storage capacity and transmission rate can be increased. In this paper, we proposed a sparse-ON pixel 4/6 balanced-modulation code that the code rate is 1.33 (bit/pixel) with uniform page density. Even though the performance of the proposed sparse-ON pixel 4/6 balanced-code is similar to 2/3 and 6/9 modulation codes, it can increase the storage capacity more than these modulation codes and also store more pages in a volume by reducing the rate of ON pixels to mitigate IPI (inter-page interference).

A Research on None Covering of Top-soil for Rice Seedling Nursery for Sparse Machine Transplanted Rice (벼 소식재배를 위한 무복토 육묘 연구)

  • Park, K.H.;Ryu, H.Y.
    • Journal of Practical Agriculture & Fisheries Research
    • /
    • v.21 no.2
    • /
    • pp.77-86
    • /
    • 2019
  • To determine none top soil covering in rice seedling nursery method for the sparse machine transplanting, four different sowing methods were tested. Shoot and root length, fresh weight, leaf number and color using leaf color chart(LCC) and SPAD were collected as the data comparison of methods. The seedling height showed the highest growth according to the conventional (230g seed rate of pre-emerged seeds and top-soil covering) > high sowing density 1 (290g seed rate of pre-emerged seeds and top-soil covering) ≥ high sowing density 2(290g seed rate of pre-emerged seeds and none top-soil covering) > high sowing density 3(290g seed rate of iron-coated seeds and none top-soil covering). There was any statistical difference between groups in root length, leaf number, LCC, and SPAD values. Thus, a high sowing density of 290g for rice nursery seedling box was recommended to the sparse machine transplanting in rice cultivation with the none top-soil covering method, enabling convenient handling in transportation and machine transplanting work.