Rate Allocation for Block-based Compressive Sensing

블록기반 압축센싱을 위한 율 할당 방법

Abstract

Compressive sensing (CS) has drawn much interest as a novel sampling technique that enables sparse signal to be sampled under the Nyquitst/Shannon rate. By noting that the block-based CS can still keep spatial correlation in measurement domain, this paper proposes to adapt sampling rate of each block in frame according to its characteristic defined by edge information. Specifically, those blocks containing more edges are assigned more measurements utilizing block-wise correlation in measurement domain without knowledge about full sampling frame. For natural image, the proposed adaptive rate allocation shows considerable improvement compared with fixed subrate block-based CS in both terms of objective (up to 3.29 dB gain) and subjective qualities.

희소성이 높은 신호를 압축센싱을 할 경우 기존의 Nyquist/Shannon 이론을 바탕으로 하는 샘플링 방법 보다 낮은 측정율 만으로도 신호의 복원이 가능하기 때문에 이를 활용한 많은 응용 연구가 이루어지고 있다. 영상신호의 경우 특히 블록기반 압축센싱 기법이 주로 고려되고 있는데, 대부분의 경우 측정 영역에서의 공간적 유사도가 동일하다는 가정 하에, 각 블록에 동일한 측정율을 할당하여 왔다. 이를 개선하기 위해, 본 논문에서는 프레임 내의 각 블록에 대하여 경계선 정보를 구하고, 각각의 특성에 따르는 적응적 샘플링율 기법을 제안한다. 제안하는 방법은 측정영역에서의 블록 간 유사도를 구해서 경계선 정보를 많이 포함하는 블록일수록 많은 측정율을 할당한다. 실험 결과, 자연영상에 대해 제안하는 적응적 율 할당 기법은 고정 측정율을 사용한 기존 방법에 비해 객관적 (최대 3.29 dB 향상) 및 주관적 화질이 뛰어나다는 것을 보여준다.

I . Introduction

Nyquist/Shannon sampling theorem is the basis for signal acquisition in digital signal processing, which tells that a band-limited signal should be sensed at a rate of being at least twice of the signal bandwidth for perfect reconstruction. However, for high dimensional signal, according to huge amount of samples, the cost of sampling and encoding thereafter might be impractical for some applications having limited sampling/encoding resources, such as surveillance camera or satellite imaging. Alternatively, a new signal sampling technique named compressive sensing (CS) overcomes this problem by sensing a sparse signal containing very few non-zero elements at a sub-Nyquist sampling rate while still guaranteeing exact reconstruction with sufficient amount of measurements.

In CS for image/video, block-based CS (BCS) scheme, where non-overlapped blocks are sensed separately, requires much less memory for storing measurement matrix; as a result, it deems to be more practical. However, conventional BCS considers all blocks equally, i.e., all blocks are sampled at the same subrate, disregarding the fact that blocks in a frame with different sparsity level (due to their different contents) may need different number of measurements for reconstruction. The subrate is the ratio of sample numbers used with and without compressed sensing scheme, so it is a number between 0 and 1. For example, a subrate of 0.5 means that the number of sampled data is reduced by half by using compressed sensing scheme. Note that, a smooth block has more compressible representation in transform domain (e.g., Discrete Cosine Transform or Discrete Wavelet Transform) than a complex block having more edge, texture or detail; so that a smooth block needs fewer measurements than a complex block to achieve the same recovery precision.

In prior researches on adaptive CS of video [1-2], the adaptive subrate of block is conveyed from decoder to encoder via a feedback channel. However the feedback channel may not be always available in practical situation, especially in real time application due to its potentially high delay. In other approaches [3-4] for image/video, the authors proposed to assign an adaptive subrate for a block based on its edge information under availability assumption of the original image (i.e., raw data that is fully sampled) for detecting edge pixels. Here, they somehow lose the advantages of CS (CS combines the sampling and compression into one step; hence, raw data is unavailable at encoder).

In this paper, we propose a method to adapt the subrate of block for image system. Subrate of each block is assigned according to its characteristic so that complex blocks are assigned more measurement data than smooth blocks without assuming availability of original image. Note that properly designed CS system should be able to preserve signal information pretty well from spatial domain to measurement domain. Under this assumption, we examine each block for its characteristic by performing edge detection in measurement domain with neither referring to raw data nor requiring feedback channel. The way we evaluate block's characteristic in measurement domain makes adaptive CS system become more simple and practical. Eventually, for image, the subrate assigned to each block is estimated proportionally to the number of edge pixels in the given block, i.e., the more edge pixels a block contains, the higher subrate the block is assigned to. Experimental results manifest superior improvement of the proposed method compared with conventional scheme of a fixed subrate BCS.

The remainder of this paper is organized as following: Section II introduces the related works. Section III presents the proposed method of adapting subrate of BCS based on edge information. Experimental results and analyses are shown in Section IV. Finally, Section V concludes the paper.

 

II . Related works

1. Compressive Sensing

A finite-length signal x∈RN is called k-sparse if it has at most k non-zero coefficients for some k ≪ N. Compressive sensing (CS) [5] is an emerging framework which can reduce the sampling cost; it is proved that a k-sparse signal x can be recovered from far fewer measurements y∈RM (i.e., M < N ) than the Nyquist/Shannon sampling rate where the measurement vector y is a linear projection of the signal x by a measurement (or sensing) matrix Փ as:

The ratio of M/N is called the subrate (or, measurement rate), which represents how much sub-sampling is done since M < N . Note that, the measurement matrix Փ needs to satisfy the restricted isometry property (RIP)[6]. Matrix Փ satisfies RIP of oder-k if there exits a δk∈(0,1) holding for all k-sparse vector x such that:

In fact, most of the natural signals are not exactly sparse, but can be sparsely represented in a proper transform domain, or compressible in some transform domains. By assumption that the signal x, through a sparse transform with its kernel ψ, can be represented as x = ψTθ, where θ is the transform coefficient of x, and the signal θ is sparse (or at least compressible); the sparse vector θ of the signal x can be recovered from the M measurements, y = Փx = ՓψTθ by l0 -minimization as following:

where the l0 -norm ∥θ∥0 is number of non-zero vector elements of θ. However, the l0 -minimization is a non-convex problem, which means potentially very difficult to solve, especially when the number of vector elements are large. Instead, CS solves (3) by l1 -minimization, which is a convex optimization problem, as:

Here, is the l1-norm of θ.

It is already known that CS can successfully recover a k-sparse signal under the condition that the number of acquired measurements M should satisfy [7]:

There are many algorithms to solve the CS reconstruction problem; for example, Matching Pursuit [8], Orthogonal Matching Pursuit [9], Multiple Candidate Matching Pursuit [10], SL0[18], or TVAL3[19] algorithms.

2. Block-based Compressive Sensing

In CS for image/video processing, the block-based CS (BCS) method has been proposed to reduce the complexity of system [11-13]. An input image is split into non-overlapping blocks denoted by x(i) ∈ RB2 where i is a block index, x(i) is a column vector with B2 elements, each of which is a pixel value inside the i-th B × B block. Then, blocks are compressively sampled into measurement domain as yi = ՓBx(i) where yi ∈ RMB is a column vector measurement and ՓB is a MB × B2 measurement matrix; in this case, subrate of a block is calculated as a ratio of MB/B2. The measurement matrix of a whole frame is a block-diagonal matrix as:

For recovery, [11] suggests a procedure that combines projected Landweber iteration with smoothing in the form of Wiener filtering. The overall process of BCS sampling and SPL reconstruction are called BCS-SPL. An improvement of BCS-SPL is also presented in [12].

 

III . Rate allocation for block-based compressive sensing

It is worth noting that BCS still keeps the block correlation in spatial domain also in measurement domain [13-15]. S. Mun et al. empirically confirmed the high correlation among measurements of BCS using Lena image, which showed average correlation even larger than 0.95 [13].

Additionally, characteristic of natural image is changed from region to region. Some regions can be smooth, while others can be complex with many objects. As a result, the sparsity (or compressibility) of image is also changed with different regions. Fig. 1 shows us more clearly about this problem. From original Lena image, we extract two blocks with two different characteristics; the upper one locates in a smooth region and the lower one locates in a complex region with many edge information (see the edge image on the left side). And then, DCT transformation is applied for two those blocks. Obviously, the sorted in descending order of absolute DCT coefficients of upper one decrease much quickly than the lower one. That means the upper one (smooth block) is more compressible than the lower one (complex block). As a result, smooth block needs less measurements than complex block to achieve equally reconstructed performance.

그림 1.부드럽고 복잡한block에 대한 절대 DCT 계수의 내림차순 정렬 Fig. 1. Sorted descendance of absolute DCT coefficients of smooth and complex block.

Motivated by the correlation among BCS measurements, we detect edges in image using the measurement data only. That is, if the block measurement data of blocks at an initial subrate s0 (i.e., same original subrate for all blocks) Y = (y1,y2,...yi,...) is block-averaged as:

where yi is the measurement vector of a i-th block , and is a mean value of measurement vector yi. Then is reshaped into a form of 2D image we can see a roughly estimated original image in a low resolution (i.e., still has good spatial correlation among blocks in measurement domain). Subsequently, the reshaped image is scaled up to original image size by a bicubic interpolation [16] where up(*) is the bicubic interpolation operator. Interestingly, the interpolated image still clearly contains edges and boundaries of the original image, and therefore, a typical edge detection algorithm is expected to identify those edge pixels well.

Fig. 2 shows the whole process of edge detection in measurement domain. Clearly, the detected edge image shows edges quite well. Thus, it is possible to characterize blocks based on the edge information, for example, a smooth block will contain very few edge pixels while a complex block will contain many edge pixels. By this way, it is expected that the more complex (less compressible) the block is, the more edge pixels the block contains. Therefore, in this paper, we estimate the sparsity of i-th block proportionally with number of edge pixels in block as:

그림 2.측정 영역에서의 경계선 검출 (측정율 0.4에서 512x512 해상도 및 8x8 블록 크기를 갖는 Lena 영상) Fig. 2. Edge detection in measurement domain (512x512 Lena image with block size 8x8, subrate 0.4)

The bigger ei means that block is less compressible. On the other hand, the number of required measurements for CS reconstruction depends much on the sparsity of block as shown in (5); hence, a sparser block needs fewer measurements [1]. Thus we should look for a way to sample each block with its proper subrate adaptively. In this paper, the rate allocation for each block is based on the edge image - blocks containing more edges (i.e., more complex) are sampled with a higher subrate than otherwise.

The whole proposed compressed sensing system is shown in Fig. 3; sparsity estimation is equivalent to edge detection process which determines the edge information in each block. The subrate of each block is assigned proportionally to spasity of block as following:

그림 3.제안하는 적응적 BCS 방법에 대한 블록 다이어그램 Fig. 3. Diagram of proposed adaptive BCS for image

where si is an adaptively selected subrate for i-th block; sm is the minimum subrate of block. sm is firstly setup sm = r × s0, (r is predefined parameter to determine how small the minimum subrate assigned for each block is, 0 < r < 1). The meaning of sm is to guarantee that the subrate assigned for each block is not so small compared with s0 which can make unstably or inaccurately in reconstruction and we may not get the improvement. Parameter α is a control parameter (its value is calculated below)

The subrate of each block is adjusted so that the final subrate of whole image is not changed compared with s0 such that:

where L is the number of blocks in image . From (8) and (9):

The value of α in (11) is then substituted into (8) to calculate subrate of each block. However, if the assigned subrate for a block exceed 1 (which is nonsense in CS system since number of measurements is not higher than signal length), sm is adjusted by increasing r (note that r is still smaller than 1), and then subrate assigned for each block is re-calculated.

As a result, a smooth block may be assigned a smaller subrate than s0 while a complex block is assigned a higher subrate than s0. Finally, since smooth blocks are assigned subrate lower than s0, we only take required number of measurement B2 × si from yi. Otherwise, those complex blocks are assigned with higher subrate than s0, we have to sample more B2 × (si-s0) measurements.

 

IV. Experimental results

In this section, performance of the proposed method is presented using four 512x512 gray natural images, namely, Lena, Barbara, Boat, and Cameraman. At encoder, the input image is split into small non-overlapped blocks and then mapped into measurement domain by an i.i.d. random Gaussian. In our experimental results, we tested three subrates 0.2, 0.4, 0.6 with two block sizes of 8x8 and 16x16. Performance of the proposed method is evacuated using three reconstructed algorithms: Iterative Support Detection (ISD) [17], Smoothed L0 Norm (SL0) [18], and total variation (TVAL3) [19]. For edge detection, Canny edge detection algorithm is used considering its good performance compared with other edge detection algorithms [20]. Additionally, value of 2/3 is assigned for the initiation of parameter r. More detail result of the experiments is shown in Table 1, Table 2, Table 3 and Fig. 4.

표 1.블록크기 8x8에서의 여러 CS복원 방법에 대한 실험 결과 [PSNR in dB] Table 1. Experimental results with various CS recovery methods for block size of 8x8 [PSNR in dB]

표 2.블록크기 16x16 에서의 여러 CS 복원 방법에 대한 실험 결과[PSNR in dB] Table 2. Experimental results with various CS recovery methods for block size of 16x16 [PSNR in dB]

표 3.여러 CS 복원 방법에 대한 실험 결과 [SSIM index] Table 3. Experimental results with various CS recovery methods [SSIM index]

그림 4.블록크기 8x8 및 측정율 0.4에서 TVAL3 기반 복원 알고리즘을 통해 산출된 영상 간 화질 비교 Fig. 4. Visual quality comparison using TVAL3 CS recovery algorithm with block size 8x8. subrate 0.2

Table 1 shows performance of various CS recovery method with a block size of 8x8. Clearly, the proposed method gains remarkably compared with the case of fixed subrate BCS. Those images with much detail, like Barbara, show PSNR improvement of 0.64 dB to 1.42 dB. With smooth images, like Cameraman, the improvement is up to 3.29 dB (subrate 0.2 with ISD algorithm).

Additionally, Table 2 shows the performance of CS recovery with block size of 16x16. Obliviously, performance of the proposed method is better than the fixed subrate BCS; however, improvement of the proposed method with block size of 16x16 is lower than that with block size of 8x8. It is because the block size of 8x8 is better in keeping the block correlation and the spatial information is also preserved well in measurement domain than the block size of 16x16. The proposed adaptive subrate assignment for each block becomes more accurate leading to higher performance in CS recovery.

Moreover, Table 3 shows visual comparison between the fixed subrate BCS and the proposed method by SSIM index with block size 8x8; it is obvious that, in all cases, the proposed method presents better performance. For more clearly, Fig. 4 illustrates an example of cropped Lena image at subrate 0.2 and TVAL3 algorithm. Fig. 4(c) shows better in visual quality (the fine detail becomes much clearer) compared with Fig. 4(b), especially in some complex regions which contains much edge information (subrate for those blocks in this region is high) leading to good reconstructed block both in subjective and objective qualities.

 

V. Conclusion

In this paper, an adaptive rate allocation for BCS of images is proposed. The characteristic of block is defined based on edge detection and then subrate of each block is assigned commensurably to how many edge pixels it contains in the edge image - blocks containing more edges are assigned more measurement data than others. The experimental results showed remarkable improvement of the proposed method compared with the fixed subrate BCS.