An Improved Level Set Method to Image Segmentation Based on Saliency

Abstract

In order to improve the edge segmentation effect of the level set image segmentation and avoid the influence of the initial contour on the level set method, a saliency level set image segmentation model based on local Renyi entropy is proposed. Firstly, the saliency map of the original image is extracted by using saliency detection algorithm. And the outline of the saliency map can be used to initialize the level set. Secondly, the local energy and edge energy of the image are obtained by using local Renyi entropy and Canny operator respectively. At the same time, new adaptive weight coefficient and boundary indication function are constructed. Finally, the local binary fitting energy model (LBF) as an external energy term is introduced. In this paper, the contrast experiments are implemented in different image database. The robustness of the proposed model for segmentation of images with intensity inhomogeneity and complicated edges is verified.

1. Introduction

Image segmentation is the extracting process of the regions of interest from an image. The results of image segmentation are widely used in traffic control, satellite positioning and medical diagnosis. Therefore, the study of image segmentation method has far-reaching practical significance.

Osher and Sethian [1] proposed the level set method in 1988. The change process follows the thermodynamic equation under the flame shape is solved by level set method. The Chan-Vese (CV) model was proposed by Chan and Vese [2]. The level set method is introduced into the active contour model and via minimizes the energy functional to drive curve. The CV model is robust to noise, but it is not ideal for the intensity inhomogeneity image segmentation. Aiming at the existing problems of CV model, a novel model (LBF) was proposed, which uses local information of image [3]. Based on the LBF model, the local image fitting energy model (LIF) was presented. In the LIF model [4], the objective energy function is constructed by using the difference between the local fitting image and the original image, which improves the efficiency of the LBF model. And the LIF model has the same good effect as LBF on the intensity inhomogeneity images. Pan et al. [5] developed an active contour model by using local entropy. This model uses the kernel function to obtain the information of the local region, and constructs the entropy energy function. This model can well describe the intensity inhomogeneity degree of the local area of the image, and it has a good effect on the image segmentation. Wang and Pan [6] exploit the entropy criterion, the sample points can be adaptively added to the cluster and the approach has good robustness to outliers. Jiang et al. [7] proposed a local entropy active contour model. By calculating the gray level distribution and the local entropy of the image, the local fitting energy of the image is got. Next, the LIF model is combined to segment images. Li et al. [8] proposed a distance preserving level set method, which added an internal energy to the traditional level set energy function. The problem that the traditional level set needs re-initialization is avoided. However, the method has a slow evolution speed and some limitations for weak edge detection. To solve the above problems, He et al. [9] introduces the weight coefficient and the new stopping function to accelerate the evolution speed and improve the ability of detecting weak edges. However, this method is lack of flexibility for the definition of weight coefficient, and false edges often appear in the detection.

In recent years, several new level set methods were proposed. Min et al. [10] proposed a dual minimization based level set method. He et al. [11] presented a novel segmentation algorithm by using the level set and improved-variation smoothing. Liu et al. [12] proposed an improved edge-based level set method combing local regional fitting information. Zhou et al. [13] proposed a correntropy-based level set method.

Aiming at the image with complicated edges and intensity inhomogeneity, an image segmentation model based on local Renyi entropy is proposed. First, the saliency map of the original image is extracted via the saliency region detection algorithm based on global contrast [14]. The contour of the saliency map is calculated according to the Otsu method and the morphological method. And the contour is taken as the initial contour of the level set, which overcomes the problem that the level set method is sensitive to the initial contour. Then, the edge energy and local energy of the image are obtained by using the Canny operator and the local average Renyi entropy, and the ability to segment the intensity inhomogeneity image is improved. At the same time, the weight coefficient and the indication function of the literature [8] are redefined, which makes the curve evolve to the edge of the image more accurately. Finally, the proposed model is used to segment the anisotropic filtering images, and more accurate segmentation results are obtained. The brain images include the problems of intensity inhomogeneity and weak edge. Due to this issue, the method of double-curve evolution is proposed in this paper. The contour obtained by the saliency map is used as the outer contour, and a circle with R radius is initialized as the inner contour. Then, the proposed model can implement double-curve evolution.

These main contributions of our method can be described as follows.

1. As we know, the level set is sensitive to the initial contour. In order to solve this issue, the initial contour is extracted by combining the saliency method, which is very close to the object edge. Not only did the sensitivity problem of initial contour is addressed, but the running time of curve evolution to the object edge is reduced.

2. The segmentation ability of the intensity inhomogeneous image is improved by integrating the local Renyi entropy and the LBF method.

3. Combined the Canny operator with the improved boundary indication function, the detection ability of the complicated object edges is improved, and the problem of missing edges is addressed.

4. In this paper, the double-curve evolution method is utilized to solve the intensity inhomogeneous problem and weak edges problem to brain image.

In the following paragraphs, Section 2 reviews some relevant level set method. Section 3 brings forward an improved level set method to image segmentation based on saliency. Section 4 gives experiment results and analysis. Finally, we conclude the paper in Section 5.

2. Background

2.1 CV Model

CV model is a piecewise constant model proposed by Chan and Vese [2] in 2001, which based on the framework of variational level set. Assuming that the known image is I(x,y), the energy functional of the CV model is shown in Eq. (1):

\(E=\lambda_{1} \int_{\text {linside}(c)}\left(I(x, y)-c_{1}\right)^{2} d x d y+\lambda_{2} \int_{\text {ouside }(c)}\left(I(x, y)-c_{2}\right)^{2} d x d y+\mu \cdot l\)       (1)

where λ₁, λ₂ and μ represent constant coefficient, the average gray value of the object area and background area of the image I(x, y) is represented by _c1 and _c2.

2.2 LBF Model

In this model [3], Gauss kernel was used to obtain the local information of the image, and segmentation results of the intensity inhomogeneity image were ideal. The energy functional of the LBF model is shown in Eq. (2):

\(\begin{array}{c} E=\lambda_{1} \int\left[\int K_{\sigma}(x-y)\left|I(y)-f_{1}(x)\right|^{2} H(\phi(y)) d y\right] d x+ \\ \lambda_{2} \int\left[\int K_{\sigma}(x-y)\left|I(y)-f_{2}(x)\right|^{2} \cdot(1-H(\phi(y))) d y\right] d x+ \\ v \cdot L(\phi)+\mu \cdot P(\phi) \end{array}\)       (2)

where λ₁, λ₂, μ and ν represent constant coefficient, f₁ and f₂ represent the values of gray fitting inside and outside of the evolution curve, respectively. K_σ represent Gauss kernel function of the standard deviation σ. The second items form fitting energy of LBF, L(Φ) and P(Φ) respectively indicates the length of regularization and the penalty.

2.3 LIF Model

This model adopted the degree of difference between the local fitting image and the original image to construct the energy function. The energy functional of the LIF model is shown in Eq. (3):

\(E=\frac{1}{2} \int_{\mathrm{a}}\left|I(x)-I^{L T}\right|^{2} d x+v \cdot L(\phi)+\mu \cdot P(\phi)\)       (3)

where μ and ν represent constant coefficient, L(Φ) and P(Φ) respectively indicates the length of regularization and the penalty, I ^LFI express local fitting image.

3. Saliency Level Set Model based on Local Renyi Entropy

3.1 Determine the Initial Contour

In general, the selection of the initial contour often determines the efficiency of algorithm. In the LBF model and the LIF model, the initial contour is initialized to a circle. The radius of the circle needs to be artificially set according to the size of the image to be segmented, which leads to the setting of the initial contour lack flexibility. In this paper, the saliency map is extracted by using the saliency detection algorithm based on global contrast [14] (Figs. 1 and 2).

Fig. 1. The original color image.

Fig. 2. The saliency map.

As the above saliency map shown, the method proposed in [14] can well detect the saliency region. Next, the Otsu method and the morphological method are used to extract the initial contour as shown in Fig. 3.

In this paper, the initial contour is automatically determined by the saliency map. The iteration number of the level set is reduced, and the curve can rapidly evolve to the object edge. Accordingly, implement efficiency of the level set is improved.

Fig. 3. Initial contour.

3.2 Construct the Level Set Energy Functional

Local information and boundary information are obtained by the local average Renyi entropy and Canny operator. The total energy functional is shown in Eq. (4):

\(E(\phi)=n E_{\operatorname{sar}}+(1-n) E_{e}+\mu \cdot P(\phi)+v \cdot L(\phi)\)       (4)

The first term represents the local fitting energy term. The local fitting energy of image is obtained by Gauss kernel function. The second term is edge energy, which includes the adaptive weight coefficient and the boundary indication function. The evolution direction of the curve and the stopping condition of the curve are determined. The third term is the penalty term, which can avoid the re-initialization of the signed distance function. The last term is the length term, in order to smooth the level set curve and avoid the isolated area.

3.2.1 Adaptive weight coefficient based on local Renyi entropy

Renyi entropy is the general form of Shannon entropy proposed by Renyi et al [15]. Its definition is shown in Eq. (5):

\(R(A)=\frac{1}{1-\alpha} \ln \sum P(i)^{\alpha}\)       (5)

Compared with the Shannon entropy, the Renyi entropy is introduced into an adjustable parameter α. Therefore, the measurement of information quantity is more general and flexible. The information entropy can be used to describe the uniformity of the regional information. If the region is more uniform, the information entropy is bigger. At the begining, we caculate the Renyi entropy of the image. Then, local information of the image is obtained by using local average Renyi entropy with window size of 3×3. Finally, the gradient of the image is calculated. The adaptive weighting coefficients is defined by

\(w(I)=\operatorname{sign}\left(\Delta I_{\sigma}\right) m \cdot\left|\nabla I_{\sigma}\right| * \Delta E_{a}\)       (6)

where I_σ express the image which is smoothed by the standard deviation σ of Gauss filtering. m>0 is the weight of local Renyi entropy whose range distribute in [5, 10]. \(\nabla\) is gradient operator. ΔE_a represent the local average Renyi entropy obtained by mean filter with window size of 3×3.

w(I) is determined by the local average Renyi entropy and gradient of the image. In the uniform regions of image, the evolution speed of the curve is accelerated because the value of local average Renyi entropy, the gradient value and the value of w(I) are all small. In the edge region of image, the value of local average Renyi entropy, the gradient value and the value of w(I) are all large. Hence, the detection ability of weak edges is improved.

3.2.2 Boundary indication function based on Canny operator

The boundary indication function [8] is improved, and the Canny operator is introduced to make the curve evolution more accurately. The boundary indication function is defined by

\(f_{e}=1-\frac{1}{\exp \left(\frac{\beta *\left|\nabla I_{\sigma}\right|}{m \cdot \Delta E_{e}}\right)}\)       (7)

where I_σ express the image which is smoothed by the standard deviation σ of Gauss filtering. m is the weight of local Renyi entropy. ΔE_a represent the local average Renyi entropy obtained by mean filter with window size of 3×3. \(\nabla\) is the gradient operator. The value of β is determined by the detection results of the Canny operator, as shown in the formula (8):

\(\beta=\left\{\begin{array}{l}1 \text { When the Canny operator does not detect the edge } \\ 0 \text { When the Canny operator detects the edge }\end{array}\right.\)       (8)

When the curve is located in the uniform region, the result of the Canny operator is not the edge, that is, β=1. At this point, the boundary indication function is greater than zero, and the curve keeps evolution. When the curve is located at the boundary of the object, the result of the Canny operator is edge, that is, β=0. At this point, the boundary indication function is equal to zero, and the curve stops evolution.

3.3 Final Energy Functional

According to the Eqs. (6) and (7), the edge energy can be expressed as

\(E_{e}=\int_{\Omega} w(I) f_{e}(\nabla I) H(-\phi) d x d y\)       (9)

In the study, the penalty term and the length term are defined by

\(P(\phi)=\frac{1}{2} \int_{\Omega}(|\nabla \phi|-1)^{2} d x d y\)       (10)

\(L(\phi)=\int_{\Omega} f_{e}(\nabla I) \delta(\phi) d x d y\)       (11)

Eqs. (2), (9), (10) and (11) are introduced into Eq. (4). The total energy functional of the model is obtained as shown in Eq. (12):

\(\begin{aligned} E=& \alpha\left[\int\left[\int K_{\sigma}(x-y)\left|I(y)-f_{1}(x)\right|^{2} \cdot H(\phi(y)) d y\right] d x+\right.\\ &\left.\int\left[\int K_{\sigma}(x-y)\left|I(y)-f_{2}(x)\right|^{2} \cdot(1-H(\phi(y))) d y\right] d x\right]+\\ & \beta \int_{\Omega} w(I) f_{e}(\nabla I) H(-\phi) d x d y+\frac{1}{2} \mu \int_{\Omega}(|\nabla \phi|-1)^{2} d x d y+\\ & \quad v \int_{Q} f_{e}(\nabla I) \delta(\phi) d x d y \end{aligned}\)       (12)

where α=1 represent the weight coefficient of the local item, β=1 is the weight coefficient of the edge item. H(·) denote the Heaviside function and δ(·) denote the Dirac Delta function.

\(H_{\varepsilon}(x)=\frac{1}{2}\left[1+\frac{2}{\pi} \arctan \left(\frac{x}{\varepsilon}\right)\right]\)       (13)

\(\delta_{\epsilon}(x)=H_{\epsilon}^{\prime}(x)=\frac{1}{\pi} \frac{\varepsilon}{\varepsilon^{2}+x^{2}}\)       (14)

The energy functional is minimized by the standard gradient descent flow. The final evolution equation is shown as follow.

\(\begin{aligned} \frac{\partial \phi}{\partial t}=& \alpha\left[\delta(\phi)\left(-\lambda_{1} e_{1}+\lambda_{2} e_{2}\right)\right]+\beta w(I) f_{e}(\nabla I) \delta(\phi)+\\ & \mu\left(\nabla^{2} \phi-\operatorname{div}\left(\frac{\nabla \phi}{|\nabla \phi|}\right)\right)+v \delta(\phi) \operatorname{div}\left(f_{e}(\nabla I) \frac{\nabla \phi}{|\nabla \phi|}\right) \end{aligned}\)       (15)

where λ₁, λ₂, μ and ν express constant coefficient and these four parameters are equal to 1. And e₁ and e₂ represent the energy terms both inside and outside the curve fitted by the Gauss kernel, which are defined by

\(e_{1}(x)=\int_{\Omega} K_{\sigma}(y-x)\left|I(x)-f_{1}(y)\right|^{2} d y\)       (16)

\(e_{2}(x)=\int_{\Omega} K_{\sigma}(y-x)\left|I(x)-f_{2}(y)\right|^{2} d y\)       (17)

where f₁ and f₂ denote the gray fitting values in and out of the curve, which are defined by

\(f_{1}=\frac{K_{\sigma}(x) *\left[H_{\epsilon}(\phi(x)) I(x)\right]}{K_{\sigma}(x) * H_{c}(\phi(x))}\)       (18)

\(f_{2}=\frac{K_{\sigma}(x) *\left[\left(1-H_{\epsilon}(\phi(x))\right) I(x)\right]}{K_{\sigma}(x) *\left(1-H_{c}(\phi(x))\right)}\)       (19)

3.4 Anisotropic Filtering

In 1990, Perona and Malik proposed anisotropic filtering diffusion model [16], the diffusion equation can be expressed as

\(\left\{\begin{array}{c} \frac{\partial I(x, y, t)}{\partial t}=\operatorname{div}(c(|\nabla I|) \cdot \nabla I) \\ I(x, y, 0)=I_{0}(x, y) \end{array}\right.\)       (20)

where div denote divergence operator, \(\nabla\) denote gradient operator, c(|\(\nabla\)\(I\)|) denote diffusion coefficient. Because the gradient value of the image edge is large, the diffusion intensity is small. On the contrary, if the gradient value of the uniform region is small, the corresponding diffusion intensity is large. Therefore, the anisotropic filtering can remove the noise and keep the edge. In the study, the anisotropic filtering image is segmented by the proposed model, and the level set curve can effectively evolve to the edge of the image.

3.5 Algorithm Flow

The improved level set method is based on the LBF model and the saliency filtering algorithm. The algorithm flowchart is shown in Fig. 4. The algorithm is described as follows:

(1) First, the saliency map of the original image is extracted by using the saliency filtering algorithm. Then, the contour of saliency map is calculated by the Otsu method and morphology.

(2) If the input image is not a brain image, the contour of saliency map is used as the initial contour of the level set. Otherwise, a circle with radius R is initialized as an inner contour. The initial contour includes the inner contour and the contour of saliency map.

(3) According to the Eq. (20), anisotropic filtering of the original image is implemented.

(4) According to the second items of Eq. (2), the fitting energy term of LBF model E_LBF is calculated.

(5) The adaptive weight coefficient based on Renyi entropy w (I) is calculated by Eq. (6).

(6) The boundary indication function based on Canny operator f_e is calculated by Eq. (7).

(7) Updating the level set energy functional by Eq. (12).

(8) Whether the curve evolves to a steady state. If not, go to step (4). Otherwise, stop segmenting.

Fig. 4. The flow chart of the proposed model.

4. Experimental Results and Analysis

The validity and correctness of the proposed method is proved by the contrast experiments on natural images, medical images and intensity inhomogeneity images. Experimental hardware environment is Intel Core i3-2120, 4 GB and 3.3 GHz, software environment is the MATLAB 2013a under Windows 7 operating system. The contrast models include CV model, LBF model and LIF model, and we compare the segmentation quality of each model and the iteration efficiency.

4.1 Natural Images Segmentation

In our research, the natural image source is Berkeley natural image database. The segmentation results of each model are shown as follows.

In Figs. 5 and 6, the initial contour of the proposed model is more close to the object edge, which can greatly reduce the iteration times of the level set function and improve the running efficiency.

In Fig. 7, the initial contour of the proposed model is very close to the object edge with 200 iteration numbers, and the evolution speed is faster than other three models.

Fig. 5. The initial contour of level set in this paper.

Fig. 6. CV model, LBF model, and LIF model level set initial contour.

Fig. 7. The segmentation results for 200 iteration numbers: (a) CV model, (b) LBF model, (c) LIF model, and (d) the proposed model.

For the segmentation of natural images, when the curve of iteration to various models no longer evolves, the segmentation is stopped. In Fig. 8, when segments large object and intensity inhomogeneity image, the LIF model is prone to over-fitting, which leads to false segmentation. It can be seen from the second row that the edge segmentation effect of the proposed model is much higher than others. Next, our research uses the Berkeley standard image database as the standard evaluation image [17]. The Jaccard similarity coefficient, Dice similarity coefficient, Precision, and Recall are used as the objective evaluation index to compare several models [18,19]. The comparison results are shown in Tables 1–4. Assuming the input image is I, the standard image is Ig and the total number of pixels is N. And the objective evaluation index can be described as follows:

\(\operatorname{Jaccard}=\frac{\sum_{i=1}^{N}\left(I_{i} \cap I g_{i}\right)}{\sum_{i=1}^{N}\left(I_{i} \cup I g_{i}\right)}\)       (21)

\(\text { Dice }=2 *\left(\frac{\sum_{i=1}^{N}\left(I_{i} \cap I g_{i}\right)}{\sum_{i=1}^{N} I_{i}}+\sum_{i=1}^{N} I g_{i}\right)\)       (22)

\(\text { Pr ecision }=\frac{\sum_{i=1}^{N}\left(I_{i} \cap I g_{i}\right)}{\sum_{i=1}^{N} I_{i}}\)       (23)

\(\operatorname{Re} \operatorname{call}=\frac{\sum_{i=1}^{N}\left(I_{i} \cap I g_{i}\right)}{\sum_{i=1}^{N} I g_{i}}\)       (24)

Fig. 8. The final segmentation results: (a) CV model, (b) LBF model, (c) LIF model, and (d) the proposed model.

Table 1. Jaccard similarity coefficient of image

Table 2. Dice similarity coefficient of image

Table 3. Precision of image

Table 4. Recall of image

For image (c) of Table 2, the image background and the edge is relatively simple. So, the LBF method obtains better local characteristics, and can detect the object edge accurately. For image (c) of Table 1, the image background and the edge is relatively simple. So, the CV method can detect the object edge quickly. However, for images (a) and (b), there are more complicated edges. Other comparison approaches compromise the missing edges issues. In this paper, the Canny operator and boundary indication function can be utilized to detect the edges more accurately. In general, the proposed method is more effective for complicated edges of image segmentation.

This study compares the real running time of different methods on different images, and the results can be shown in Table 5.

Table 5. Running time and iteration comparison of segmentation on image different size

By comparing the running time of several models, the running time of CV is the shortest. The LBF model needs Gaussian smoothing, the LIF model needs to construct the local fitting image. In the edge detecting process, the proposed method needs to adopt Canny operator and boundary indication function. So, the computation time is costly. Yet, the running time of the proposed method is only slower than the CV model, which has better detection performance for the image with complicated edges. The proposed method only cost little running time to guarantee the segmentation performance.

4.2 Medical Image Segmentation

The medical image source is the McConnell Brain Imaging Center, which includes intensity inhomogeneity and weak edge features. In this paper, a double-curve evolution method is proposed to segment brain images. First, the contour extracted from the saliency map is used as the outer contour of the level set evolution. Then, a circle with radius R=30 initialized by signed distance function is used as the inner contour of the level set evolution. Finally, the proposed model is used to segment image.

In Fig. 9, the CV model can segment the outer edge of image, but the segmentation effect of intensity inhomogeneity image is not ideal. The LBF model improves the segmentation ability for intensity inhomogeneity image, but the segmentation effect of the weak edge is lower than the proposed model. The LIF model has a good effect on the segmentation of gray image, but the edge segmentation is jagged. The proposed model can segment the inner intensity inhomogeneity part of the brain image, and the weak edge segmentation result is ideal.

Fig. 9. The segmentation results of brain images: (a) CV model, (b) LBF model, (c) LIF model, and (d)the proposed model.

4.3 Inhomogeneous Image Segmentation

In order to further assess the performance of the proposed model, images with inhomogeneity are used to compare in different models. In Fig. 10, the first image includes severe intensity inhomogeneity and shade. The second and third images are vessel blood images with severe intensity inhomogeneity and low contrast.

Fig. 10. The initial contour of different models: (a) comparison models and (b) the proposed model.

From Fig. 11 it can be analyzed that CV model produces less-segmentation because it only extracts global image information. The Gauss Kernel is used to obtain local image information in the LBF model which improves segmentation results. LIF model uses local fitting image to get local image information, which can enhance segmentation performance. However, the evolution process of the LIF model is not steady, which causes jagged edge. In this paper, the Gauss Kernel and the Renyi entropy are used to extract local image information. Moreover, the improved boundary indication function is used to obtain image edge information. The final segmentation results of images validated that the proposed model had good segmentation performance.

Fig. 11. The inhomogeneous image segmentation results of different models: (a) CV model, (b) LBF model, (c) LIF model, and (d) the proposed model.

5. Conclusion

In this paper, an improved level set method to image segmentation based on saliency is proposed. For the level set energy function, this paper introduces the local average Renyi entropy and Canny operator to obtain the local information and edge information of the image. By combing with the LBF model, the energy function of the proposed model is constructed. This paper mainly has the following three advantages. (1) The initial level set contour obtained by the saliency map is very close to the object edge, which reduces the iteration numbers of the level set function. (2) We use the local average Renyi entropy and Canny operator to obtain the local information and the edge information of the image. What’s more, the detection ability of the intensity inhomogeneity image and the weak edge image is greatly improved. (3) The intensity inhomogeneity and weak edge of brain image are hard to segment by single curve evolution. Therefore, we propose the double-curve evolution method to segment brain images.