Low-Rank Representation-Based Image Super-Resolution Reconstruction with Edge-Preserving

Abstract

Low-rank representation methods already achieve many applications in the image reconstruction. However, for high-gradient image patches with rich texture details and strong edge information, it is difficult to find sufficient similar patches. Existing low-rank representation methods usually destroy image critical details and fail to preserve edge structure. In order to promote the performance, a new representation-based image super-resolution reconstruction method is proposed, which combines gradient domain guided image filter with the structure-constrained low-rank representation so as to enhance image details as well as reveal the intrinsic structure of an input image. Firstly, we extract the gradient domain guided filter of each atom in high resolution dictionary in order to acquire high-frequency prior information. Secondly, this prior information is taken as a structure constraint and introduced into the low-rank representation framework to develop a new model so as to maintain the edges of reconstructed image. Thirdly, the approximate optimal solution of the model is solved through alternating direction method of multipliers. After that, experiments are performed and results show that the proposed algorithm has higher performances than conventional state-of-the-art algorithms in both quantitative and qualitative aspects.

1. Introduction

The high resolution (HR) images contain more critical information and rich details, which are widely applied in smart phone camera, remote sensing and object detection. However, since the limitation of camera and the influence of external imaging environment, the resulting images may lose some critical details and the image resolution is lower. Single image super-resolution (SISR) methods, as an effective image processing technology, can produce an excellent HR image from an observed low-resolution (LR) image. In recent years, SISR has become an active topic. Many researchers study various reconstruction methods. The existing methods may generally be classified into three types: interpolation-based approaches [1,2], reconstructed-based approaches [3-11], and learning-based approaches [12-20]. From the perspective of the quality and the speed of the reconstructed image, learning-based approaches reveal their prominent advantages in all SR reconstruction methods. Therefore, in this paper, we focus on the learning-based approaches

Learning-based or example-based SR methods utilize a learned database consisting of LR and HR image patch pairs to derive the mapping between LR and HR feature spaces, estimating the HR image [21]. According to the establishment of mapping relationship, typical example learning-based SR methods mainly include manifold learning-based approaches [14, 22-24], example regression-based approaches [17, 18, 25], deep learning-based approaches [26-28] as well as sparse representation-based approaches [13, 15, 29, 30]. For example, Chang et al. [14] propose the neighbor embedding (NE) algorithm as the representative of the manifold learning-based methods. This algorithm assumes that the local geometry of the nonlinear manifolds formed by LR image patches is similar to that of their HR counterparts. Timofte et al. [17] propose the anchored neighbor regression (ANR) method, and then on this basis, they combine the best qualities of ANR algorithm and simple functions (SF) to obtain the adjusted anchor neighbor regression (A+) algorithm [18]. Dong et al. propose a model named super-resolution deep convolutional neural network (SRCNN) [26] consisted of three convolutional layers, which directly learns an end-to-end mapping between low- and high-resolution images. Assuming that each LR image patch shares the same sparse coefficient as its corresponding HR patch, Yang et al. utilize the sparse coefficient and the trained HR dictionary to generate a HR image [13]. Dong et al. [15] build a novel adaptive sparse domain selection (ASDS) scheme by integrating local autoregressive (AR) and nonlocal self-similarity (NLSS), which performs well on image deblurring and SR reconstruction. In [30], Huang et al. firstly introduce the gradient domain guided filter [31] into the ASDS scheme, and obtain a novel robust image super-resolution method to preserve edges.

Recently, many researchers have shown great interest in the low-rank representation (LRR) and apply it into image SR reconstruction [3], [32-39], data clustering [40-42], and other fields. In [32], Chen et al. introduce the low-rank matrix recovery (LRMR) technique into the neighbor embedding (NE) SR method and obtain excellent results. Zhao et al. [33] investigate the LRR combining the sparsity with the correlation to establish an adaptive sparse coding-based super-resolution (ASCSR) model. In [35], Lu et al. study the locality-constrained low-rank representation (LLR) and apply it into a unified representation-based face SR framework, constructing HR image. Some low-rank based methods may capture the global structure features of an image and exhibit the powerful SR results. However, these methods have disadvantages, such as blurring edge structures and destroying critical details, since there doesn’t exist enough similar patches for any exemplar patch [36,37]. In order to solve these problems, inspired by [30] and [35], we propose a representation-based image super-resolution method introducing the gradient domain guided image filter into the LRR scheme. The aim is to reveal the essential structure of an image, and simultaneously preserve its edges during the super-resolution reconstruction.

The major contributions of this paper are:

(1) The prior information of the HR dictionary atom is introduced into the low-rank representation scheme via the gradient domain guided image filter, which can full use of the external high-frequency information to enhance image details.

(2) An effective optimization model is established, which combines the global and local structure information to reveal the intrinsic structure of the input image and simultaneously preserve the edges.

(3) The alternating direction method of the multiplier (ADMM) [43] is used to calculate the approximate solution of the proposed optimization problem, so as to get its representation coefficient.

The rest of the paper is organized as follows. In section 2, we summarize related work on the sparse representation and the low-rank representation. In section 3, the proposed model is described and analyzed in detail. Experiments are performed to compare the proposed model with several conventional methods in section 4. Conclusions are given in section 5.

2. Related Work

In this section, we briefly review the related theories for SR problem, including sparse representation in SISR and low-rank representation, which are important to our proposed model.

2.1 Sparse Representation in Single Image Super-Resolution

The degradation process of image observation is expressed as follows

Y = SHX + v,       (1)

where X and Y respectively represent the original HR image and observed LR image. H is the blurring operator, and S is the down-sampling operator, v represents noise, which is defined as additive Gaussian noise. Sparse representation can effectively tackle the above inverse problem

In [13], Yang et al. firstly propose sparse representation-based SR method, they exploit the joint dictionary training strategy to get the HR dictionary D_h and the LR dictionary D_l , thus the two dictionaries share the same sparse representation coefficeient. After that, the sparse representation coefficient vector can be obtained through solving the following optimization problem.

\(\min _{\alpha}\left\|\boldsymbol{D}_{l} \boldsymbol{\alpha}-\boldsymbol{y}\right\|_{2}^{2}+\lambda\|\boldsymbol{\alpha}\|_{1}\)       (2)

here y denotes the patch of LR imageY , λ ( λ > 0 ) is the regularization parameter that balances the sparsity and the error term. Once the solutionα of Eq. (2) is obtained, the corresponding HR feature patch x can be recovered as x=D_hα.

2.2 Low-Rank Representation

Given an observation matrix A , suppose that A is corrupted by errors or noises E₀ (A = D + E₀). In order to recover the low-rank matrix D from A , consider the following regularized rank minimization problem

\(\min _{D} \operatorname{rank}(\boldsymbol{D}), \text { s.t. } \boldsymbol{A}=\mathbf{\Phi D}\)       (3)

where Φ is the dictionary. More generally, replace the rank function with the nuclear norm to generate the following convex optimization expression

\(\min _{D}\|D\|_{*}, \text { s.t. } A=\Phi D \)       (4)

where ||D||, denotes the kernel norm of D that is the sum of all singular values of the matrix D .

The low-rank constraint on matrix D may uncover the A ’s intrinsic subspace structure and accurately cluster its samples. Therefore, in SR reconstruction, the LRR can uncover the global structure of the image. In the next section, the gradient domain guided filter [31] is incorporated into the LRR scheme to improve the quality of the reconstructed image.

3. Proposed Method

In this section, we detail the proposed model. Firstly, the gradient domain guided image filter is extracted from the HR dictionary, and a LRR model with edge-preserving is constructed, which balances the global intrinsic structure and the local detail enhancement. Then, the representation coefficient of the established optimization model is acquired via the alternating direction method of multiplier (ADMM) [43].

3.1 Model of the Proposed Algorithm

Inspired by Trace Lasso [44], Zhao et al.[33] propose an ASCSR algorithm by introducing a low-rank regularization term: \(||D_ldiag(\alpha)||_*\). Their mathematical form may be written as

\(\min _{\alpha}\left\|\boldsymbol{D}_{l} \boldsymbol{\alpha}-\boldsymbol{y}\right\|_{2}^{2}+\lambda\left\|\boldsymbol{D}_{l} \operatorname{diag}(\boldsymbol{\alpha})\right\|_{*}\)       (5)

Due to the characteristics α₂≤||D₁diag(α)||_* ≤ α₁, it can well embody the sparsity and the correlation of image patches. Although the ASCSR algorithm may coordinate the relationship between the sparsity and the correlation via the LRR, the traditional LRR methods cluster similar patches into the same subspace on the assumption that the subspaces are independent. That is, the LRR merely captures the global structure of the data, without taking into account the local structure information. Therefore, it is necessary to further explore the underlying local structure of the data.

The LRR-based SR reconstruction methods usually assume that there are sufficient similar patches to ensure their low-rank property. In practice, the assumption may lose critical image details and edge structure during the process of the reconstruction. Consequently, the edge-preserving is important to the SR reconstruction. By adding a constraint term to the objective function, the LRR is expanded to structure-constrained LRR (SC-LRR) [41] to restrict the structure of its solution, improving performances of the disjoint subspace segment. The gradient domain guided image filter acted as a local filter may better preserve image edges and enhance image details. As a result, we introduce the gradient domain guided image filter as an edge constraint term into the LRR to boost image reconstruction performance. It can be expressed by the following equation

\(\boldsymbol{\alpha}=\underset{\boldsymbol{\alpha}}{\arg \min }\left\{\left\|\boldsymbol{y}-\boldsymbol{D}_{l} \alpha\right\|_{2}^{2}+\lambda_{1}\left\|\boldsymbol{D}_{l} \operatorname{diag}(\boldsymbol{\alpha})\right\|_{*}+\lambda_{2}\|\boldsymbol{E} \otimes \boldsymbol{\alpha}\|_{2}^{2}\right\}\)       (6)

where α ∈ R^dx1 is the low-rank coding vector. ⊗ denotes the element-wise multiplication (the Hadamard product). λ₁ and 2_λ are the parameters that control the balance between low-rank and edge-preserving. E = [e₁, e₂, ..., e_d]^T, e_i = exp(\(\frac{\left\|\boldsymbol{y}-\hat{\boldsymbol{y}}_{G}^{i}\right\|_{2}^{2}}{\sigma} \)) , describes edge constrain,σ is a constant used to adjust the speed of weight decay, and \(\hat y^i_G\) is the gradient domain guided filter of the i -th atom in D_h dictionary, and \(\hat y^i_G\) is given by the following equation

\(\hat{\boldsymbol{y}}_{G}^{i}=\bar{a}_{p} \boldsymbol{y}_{h}^{i}+\bar{b}_{p}\)       (7)

where \(y^i_h\) is the i-th atom of D_h dictionary, \(\bar{a}_{p}=\frac{1}{\left|\Omega_{\zeta_{1}}\right|} \sum_{p^{\prime} \in \Omega_{\zeta_{1}}(p)} a_{p}\) and

\(\bar{b}_{p}=\frac{1}{\left|\Omega_{\zeta_{1}}\right|} \sum_{p^{\prime} \in \Omega_{\zeta_{1}}(p)} b_{p}\)  are the mean of a_p' and b_p' [31].

In the optimization model Eq.(6), the rank minimization of matrix D_ldiag(α) means that we may select the most accurate dictionary for the reconstructed image patch y to against noise. In [35], it is confirmed that the structure constraint of subspaces is more necessary than the sparsity so as to obtain better prior information from dictionary atom. For the purpose of taking full advantage of the high-frequency prior information in dictionary atom, we preserve edges by minimizing the difference between the gradient domain guided filter of the HR dictionary atom and the LR input image patch. Therefore, we can simultaneously exploit both the global structure information and the local structure information.

3.2 Deduce the Iterative Algorithm by ADMM

Since the proposed model (6) is convex, the iterative algorithm may be derived by ADMM [42,43] in order to solve the approximate solution. It may be rewritten as

\(\boldsymbol{\alpha}=\underset{\alpha}{\arg \min }\left\{\left\|\boldsymbol{y}-\boldsymbol{D}_{l} \alpha\right\|_{2}^{2}+\lambda_{1}\|\boldsymbol{Z}\|_{*}+\lambda_{2}\|\boldsymbol{E} \otimes \alpha\|_{2}^{2}\right\}, \\s.t.\boldsymbol{Z} = \boldsymbol {D}_l diag(\boldsymbol {\alpha}).\)       (8)

The augmented Lagrangian function of the above equation can be expressed as

\(\begin{aligned} L(\boldsymbol{Z}, \boldsymbol{\alpha}, \boldsymbol{C}, \mu) &=\left\|\boldsymbol{y}-\boldsymbol{D}_{l} \boldsymbol{\alpha}\right\|_{2}^{2}+\lambda_{1}\|\boldsymbol{Z}\|_{*}+\lambda_{2}\|\boldsymbol{E} \otimes \boldsymbol{\alpha}\|_{2}^{2}+\left\langle\boldsymbol{C}, \boldsymbol{Z}-\boldsymbol{D}_{l} \operatorname{diag}(\boldsymbol{\alpha})\right\rangle \\ &+\frac{\mu}{2}\left\|\boldsymbol{Z}-\boldsymbol{D}_{l} \operatorname{diag}(\boldsymbol{\alpha})\right\|_{F}^{2} \\ &=\left\|\boldsymbol{y}-\boldsymbol{D}_{l} \boldsymbol{\alpha}\right\|_{2}^{2}+\lambda_{1}\|\boldsymbol{Z}\|_{*}+\lambda_{2}\|\boldsymbol{E} \otimes \boldsymbol{\alpha}\|_{2}^{2}+\frac{\mu}{2}\left\|\boldsymbol{Z}-\boldsymbol{D}_{l} \operatorname{diag}(\boldsymbol{\alpha})+\frac{\boldsymbol{C}}{\mu}\right\|_{F}^{2}, \end{aligned}\)        (9)

where λ₁ and λ₂ are the parameters for balancing different regularization terms, C represents the Lagrange multiplier, <⋅,⋅> represents the inner product, µ is the penalty parameter, and ||·||_F is the Frobenius norm. Through alternately calculating each variable in Eq. (9) and fixing remainder variables, we can obtain the solution of all variables. The specific steps are as follows.

Firstly, update the low-rank matrix Z ,α and other variables are fixed. This can be solved by minimizing the following equation

\(\begin{aligned} \boldsymbol{Z}_{k+1} &=\underset{\boldsymbol{Z}}{\arg \min }\left\{\lambda_{1}\|\boldsymbol{Z}\|_{*}+\frac{\mu_{k}}{2}\left\|\boldsymbol{Z}-\boldsymbol{D}_{l} \operatorname{diag}\left(\boldsymbol{\alpha}_{k}\right)+\frac{\boldsymbol{C}_{k}}{\mu_{k}}\right\|_{F}^{2}\right\} \\ &=\Theta_{\frac{\lambda_{1}}{\mu_{k}}}\left(\boldsymbol{D}_{l} \operatorname{diag}\left(\boldsymbol{\alpha}_{k}\right)-\frac{\boldsymbol{C}_{k}}{\mu_{k}}\right) \end{aligned}\)       (10)

where Θ denotes the singular value threshold (SVT) operator acting on matrix. Defining \(\operatorname{Udiag}\left(\Psi_{\frac{\lambda}{\mu_{k}}}(\sigma)\right) V^{T}\) as the singular calue decomposition (SVD) of \(\boldsymbol{D}_{l} \operatorname{diag}\left(\boldsymbol{\alpha}_{k}\right)-\frac{\boldsymbol{C}_{k}}{\mu_{k}}\) then we get the following equation

\(\boldsymbol{Z}_{k+1}=\Theta_{\frac{\lambda_{1}}{\mu_{k}}}\left(\boldsymbol{D}_{l} \operatorname{diag}\left(\boldsymbol{\alpha}_{k}\right)-\frac{\boldsymbol{C}_{k}}{\mu_{k}}\right)=\operatorname{Udiag}\left(\Psi_{\frac{\lambda_{1}}{\mu_{k}}}(\boldsymbol{\sigma})\right) V^{T}\)       (11)

where \(\Psi_{\frac{\lambda_{1}}{\mu_{k}}}\) represents the SVT operator applying to vector σ.

Secondly, we update α_k+1 in Eq. (8), and then the derivation is written as follows

\(\begin{aligned} \boldsymbol{\alpha}_{k+1} &=\underset{\boldsymbol{\alpha}}{\arg \min }\left\{\left\|\boldsymbol{y}-\boldsymbol{D}_{l} \boldsymbol{\alpha}\right\|_{2}^{2}+\lambda_{2}\|\boldsymbol{E} \otimes \boldsymbol{\alpha}\|_{2}^{2}+\frac{\mu_{k}}{2}\left\|\boldsymbol{Z}_{k+1}-\boldsymbol{D}_{l} \operatorname{diag}(\boldsymbol{\alpha})+\frac{\boldsymbol{C}_{k}}{\mu_{k}}\right\|_{F}^{2}\right.\\ &=\boldsymbol{A} \boldsymbol{D}_{l}^{T} \boldsymbol{y}+\boldsymbol{A} \operatorname{diag}\left(\boldsymbol{D}_{l}^{T}\left(\boldsymbol{Z}_{k+1}+\frac{\boldsymbol{C}_{k}}{\mu_{k}}\right)\right) \end{aligned}\)       (12)

where \(\boldsymbol{A}=\left[\boldsymbol{D}_{l}^{T} \boldsymbol{D}_{l}+\lambda_{2} \operatorname{diag}(\boldsymbol{E})^{T} \operatorname{diag}(\boldsymbol{E})+\operatorname{diag}\left(\operatorname{diag}\left(\boldsymbol{D}_{l}^{T} \boldsymbol{D}_{l}\right)\right)\right]^{-1}\).

The Lagrange multiplier C and the penalty parameter µ are updated as follows

C_k+1 = C_k + µ_k (Z_k+1 = D_ldiag(α_k+1)),       (13)

µ_k+1 = min(ρµ_k, µ_k). 

The procedure of the approximate solution of optimization problem (6) based on ADMM is summarized in following Algorithm1. With the solution α, the HR image patch is reconstructed by x = D_hα .

Algorithm 1 An iterative algorithm based on ADMM for solving the optimization problem (6)

3.3 Single Image SR via the Proposed Method

In general, for the example-based SR algorithm, we analyze image characteristics in patches. Therefore, given a LR image Y ∈ R^mxn, to generate a HR image X ∈ R^tmxtn with the scale factort, we up-sample the image Y with an interpolation operator, extract gradient features, and divide the resulting feature image into a series of overlapped patches \(\boldsymbol{y} \in R^{\sqrt{l} \times \sqrt{l}}\). Then, we concatenate each resulting feature patch into a feature vector. To make the equation be simple, they are still expressed as y ∈ R^lx1.

In the training stage, the method described in [13] is applied to train dictionary. For each LR feature patch y, the HR counterpart x is reconstructed by x=D_hα , where the sparse coefficient α is achieved by Algorithm 1. Finally, the mean of y is added to x , and all resulting HR image patches are combined into a complete image X₀ . Furthermore, the global reconstruction constraint (1) is enforced on X₀ to obtain a more satisfactory reconstructed image, as follows

\(\boldsymbol{X}_{*}=\underset{\boldsymbol{X}}{\arg \min }\|\boldsymbol{Y}-S H \boldsymbol{X}\|_{2}^{2}+c\left\|\boldsymbol{X}-\boldsymbol{X}_{0}\right\|\)       (14)

The Eq. (14) is solved by gradient descent method, and its iterative update is written as follows

X_t+1 = X_t + v[H^T S^T (Y - SHX_t) + c(X_t - X₀).       (15)

The following Algorithm 2 summarizes how to perform single image super-resolution reconstruction via our proposed approach.

Algorithm 2 SISR via LRR and gradient prior

4. Experiments and Analysis

In this paper, experiments are all implemented on an Intel(R) Core(TM) i7-6500U PC under the Matlab R2017b programming environment. We choose several natural HR images from Set 5, Set 10, Set 14 used in [15] and [18], as shown in Fig. 1. To objectively evaluate the performance of reconstructed HR images, the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) are acted as experimental evaluation criteria.

Fig. 1. Ten HR test images, from left to right and from top to bottom: Plants, Bike, Butterfly, Flower, Girl, Parthenon, Leaves, Parrots, Raccoon, Starfish.

In the experiment, the size of the patch is set to 5×5, the overlap between adjacent patches is 1 pixel, and the magnification of the LR image is 2 or 3. For a color image, the human visual system (HVS) is more sensitive to the variation in luminance, so we apply our proposed method to the luminance channel (Y channel in YCbCr color space) and apply the bicubic interpolation to the Cb and Cr channel. All dictionaries are obtained by training 100000+ pairs of patches and their sizes are set to 1024, which is as similar as [13].

4.1 Effectiveness of Our Proposed Method

In this subsection, our method is compared with Bicubic interpolation, ScSR [13], ANR [17], ASCSR [33], SRCNN [26], ASR+LR [34], and the experimental results on ten images shown in Fig. 1 are presented. We set the identical parameter λ₁ = 0.2 as in the paper [33], and λ₂ = 0.1. The selection of the parameter λ₂ may be specifically discussed in section 4.2. And the initial values of α , Z andC are all set to zero. Table 1 and Table 2 respectively list the PSNR and SSIM results obtained by our approach and the abovementioned methods when magnification factors are 2 and 3.

Table 1. Comparison of our method with Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26] and ASR+LR[34](scaling factor s = 2).

Table 2. Comparison of our method with Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26] and ASR+LR[34](scaling factors = 3).

As listed in Table 1 and Table 2, SRCNN shows the best results. In all the sparse representation-based methods, ASR+LR achieves the best results on Flower and Girl images, and the highest SSIM values are also achieved on Plants and Parrots images, which indicates the ASR+LR method can enhance structure information. However, the PSNR and SSIM averages obtained via our reconstruction method are the highest among all the sparse representation-based methods, which fully demonstrates the effectiveness and superiority of our approach.

To visually evaluate the proposed approach, Fig. 2 shows the visual contrast results with a magnification of 2 on image Leaves, Fig. 3 and 4 show the visual detail comparison of the Plants and Butterfly images with a magnification of 3. Adding an edge-preserving regularization term, the sharp edges and rich details are well restored. Take Fig. 3 as an example, we can observe that the Bicubic interpolation generates a bit blur in the reconstructed image Plants, while ScSR and ANR have sharper outline and edges, with some ringing artifacts, ASCSR and ASR+LR have a certain improvement in visual effect. Deep learning-based methods have performed excellently in image SR, but they still have some disadvantages. For example, they always need large amounts of data for training, which are sometimes difficult to obtain. In addition, they usually take a lot of time even several days to train the network on very sophisticated graphical processing units (GPUs). In this paper, we exploit the method in [13] without requiring a great many of data and spending too much time to build a couple dictionary. To sum up, our jobs not only have higher performance to reveal the details of the image but also have better actual SR reconstruction effect.

Fig. 2. Comparison of SR results on Leaves by different methods (scaling factor s = 2 ), from left to right and from top to bottom: Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26]，ASR+LR[34], Proposed method.

Fig. 3. Comparison of SR results on Plants by different methods (scaling factors = 3), from left to right and from top to bottom: Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26], ASR+LR[34], Proposed method.

Fig. 4. Comparison of SR results on Butterfly by different methods (scaling factor s = 3), from left to right and from top to bottom: Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26],ASR+LR[34], Proposed method.

4.2 Discussion about the Parameter Settings in Our Method

The low-rank property of our method means that objective function (6) drops the edge-preserving regularization term \(\|E \otimes \alpha\|_{2}^{2}\)​​​​​​​ λ₂ = 0. Then the objective function is equivalent to the ASCSR model proposed in [33], the low-rank property and corresponding parameter settings have already been discussed in [33] and we don’t repeat here. Extracting the gradient domain guided filter of every HR dictionary atom, the high-frequency details in the HR dictionary can be utilized as the prior information. In this subsection, we mainly discuss the settings of the trade-off parameter λ₂. In Fig. 5, when λ₂ is changed from0 to 0.5, the PSNR and SSIM average curves of ten test images obtained by our proposed method are plotted, respectively. As a result, when λ₁ = 0.2 and λ₂ = 0.01, both of them reach the peak. And after λ₂ = 0.2, PSNR and SSIM tend to stable.

Fig. 5. From left to right, average effect of PSNR and SSIM values on different λ₂.

4.3 Robustness of Our Proposed Method

In practice, the images are usually contaminated by noises. Herein, we choose any six of the ten test images, which are Bike, Butterfly, Leaves, Parrots, Raccoon and Starfish to evaluate the robustness of our method against noise. Gaussian noises with a standard deviation of 2,4,6 are respectively chosen to add into the input LR images, corresponding to λ₁=0.2, 0.4 , 0.6 ; and λ₂ is a constant λ₂ = 0.1. Table 3 lists the average PSNR and SSIM values for the six reconstructed images obtained through different methods under different Gaussian noises. Although the performance of SRCNN is the best among the above algorithms, our proposed algorithm achieves the best objective results among the sparse representation-based approaches. Furthermore, in order to get better reconstruction results, SRCNN requires higher overhead during training, such as a large number of samples, more model parameters, and a lot of time. Fig. 6 and 7 illustrate the visual effect of Bike and Starfish. As we know, noises degrade the image quality. From results, the proposed method and the ASR+LR have higher performance than other methods since they may well in suppressing noises. That is the low-rank constraint helps to enhance image SR robustness.

Fig. 6. SR results on Bike by different methods (scaling factors = = 2, 6 σ ), from left to right and from top to bottom: Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26]，ASR+LR[34], Proposed method.

Fig. 7. SR results on Starfish by different methods (scaling factor s = = 3, 6 σ ), from left to right and from top to bottom: Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26]，ASR+LR[34], Proposed method.

Table 3. Average PSNR (dB) and SSIM results on noisy images (Bike, Butterfly, Leaves, Parrots, Raccoon, Starfish).

4.4 Discussion on the Performance of Noisy Real Word Image

Actual noisy images in SUN database [45] are used to verify our method in this section. Two actual images (as shown in Fig. 8) are selected, and the regions of interest (ROI) marked in the figures are reconstructed to verify the feasibility and robustness of the approach in practice. We perform SR reconstruction on the ROI areas in Airport and Building with the magnification factor3. Fig. 9 and 10 display the visual comparison of SR results obtained by our method with Bicubic, ScSR, ANR, ASCSR, SRCNN and ASR+LR. As shown in the two figures, we get the following conclusion. Firstly, the Bicubic produces the most blurred image. Then, ASCSR and ASR+LR generate better results than ScSR and ANR, since the edges of the image are cleaner. Finally, our proposed method well preserves the edges and reveals more robust against noise than other sparse representation-based methods. Compared to SRCNN, our method almost has the similar visual effect as it does. The experimental results indicate that our method can be applied to real word noisy images and meet the need of actual image reconstruction.

Fig. 8. From left to right, actual noisy images: Airport, Building and their corresponding ROI areas for testing.

Fig. 9. SR results of the ROI in Airport image by different methods (scaling factor s = 3), from left to right and from top to bottom: Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26]， ASR+LR[34], Proposed method.

Fig. 10. SR results of the ROI in Building image by different methods (scaling factors = 3), from left to right and from top to bottom: Bicubic, ScSR[13], ANR[17], ASCSR[33], SRCNN[26]， ASR+LR[34], Proposed method.

5. Conclusion

In this paper, the gradient domain guided filter of HR dictionary atoms is introduced into the LRR scheme as an edge-preserving regularization term, and a robust SISR reconstruction method is proposed. Through experimental results, the LRR is proved to effectively capture the global structure of the image. The gradient domain guided filter incorporates an explicit first-order edge-aware constraint to enhance the fine detail of an image based on local optimization. The proposed method integrates the edge-preserving regularization term into the LRR, which can simultaneously exploit both the global and local structure of the image to ensure the quality of the restored HR image. Extensive experiments on test images demonstrate that the proposed approach is more competitive than some other state-of-the-art methods.