Color Image Enhancement Using a Retinex Algorithm with Bilateral Filtering for Images with Poor Illumination

Abstract

Color enhancement basically deals with color manipulation in digital images. Recently, the technique has become widely used as a result of the increasing use of digital cameras. Retinex-based colorenhancement algorithms are a popular technique. In this paper, retinex with bilateral filtering is proposed to improve the quality of poorly illuminated images. Generally, it consists of three main steps: first, a retinex-based algorithm with color restoration; second, transformation mapping using histogram matching; and finally, smoothing the image using a bilateral filter. The experimental results demonstrate that the proposed method can successfully enhance image contrast while avoiding the halo effect and maintaining the color distribution in the image.

1. INTRODUCTION

Over recent decades, image enhancement has become an important research field for improving the quality of images. It focuses on processing an image to bring out important hidden features. Generally, medical microscopy and real-world images often suffer from poor contrast or lack of detail [1-4]. Thus, image-enhancement techniques are used to manipulate the original images to obtain better results for further image processing and analysis.

Recently, many different techniques have been used. Histogram equalization (HE) is a well-known contrast-enhancement technique, because of its strong performance for almost all types of image [5-13]. A homomorphic filtering algorithm is used to provide enhancement in an image by deemphasizing the illumination effect and emphasizing higher frequencies of the image; it can be applied by setting apart these two elements of the image [14]. The unsharp masking algorithm is used to sharpen an image by adding a mask to the original image. The mask is obtained by subtracting the blurred image from its original. However, this creates a halo effect, which is caused by under- and over-enhancement. Thus, a modified unsharp masking method was proposed [15]; it overcame this drawback using an edge-preserving filter and adding a rescaling process.

The retinex theory assumes that image intensity is the product of illumination and reflectance [16]. This theory has been used in many fields, such as aerospace, safety and security, consumer imagery, science and technology, medical purposes, and even in forensics. New algorithms based on retinex theory have been developed, such as single-scale retinex (SSR) [17], multi-scale retinex (MSR) [18], and multi-scale retinex with color restoration (MSRCR) [19-20].

In the SSR method, the illumination value is approximated by applying a Gaussian surround function to an input image. The retinex output in the ith color channel is shown in the following equation:

where i ∈ {R, G, B}, Ii(x, y) is the input image intensity value in the ith color channel, * represents the convolution operation, and F(x, y) is the Gaussian surround function.

However, while SSR dramatically improves image quality, it also causes a halo effect due to overenhancement. MSR was proposed as a solution to this problem. Its output is a linear combination of SSR output images using Gaussian surround functions with different sizes. Because SSR and MSR are applied individually to each color channel of the red, green and blue (RGB) image, changes in color are observed in the resulting image. Thus, MSRCR was proposed to keep the ratio of R, G, and B components in the input image. However, the result often still looks unnatural.

Previous works [21-23] provide experiments and brief explanations related to enhancing the naturalness of images. Those studies took original ideas from retinex theory, and modified the process.

In this paper, we present an algorithm to simultaneously enhance images and preserve their naturalness. The retinex-based algorithm is applied as an image-quality improvement process. However, the resulting distribution of intensity still does not look good, so transformation mapping is applied. Then, in the final step, a bilateral filter is used to reduce the noise while preserving the edges of the image.

The remainder of this paper is organized as follows. Part 2 details the proposed method and basic theory regarding the method. Part 3 presents the experimental results compared to other types of enhancement methods. Finally, Part 4 provides our conclusions.

 

2. PROPOSED METHOD

Our proposed method consists of three main steps. First, the image-quality improvement process uses a multi-scale retinex with a CRparameter. It is applied independently to each R, G, and B color channel of an input image. The aim of this process is to restore the details and contrast of the image. Second, transformation mapping is applied to the image. Third, an edge-preserving smoothing algorithm is performed to produce the final enhanced image.

2.1 Image-quality improvement

The output of the MSR at each color channel is as follows:

where i ∈ {R, G, B}, (x, y) are the pixel coordinates, Wa is the weighting factor, Fa is surround function, and A is the number of surround functions. Fa is given as:

where σa is the standard deviation and β is a normalizing factor. The magnitude of σa controls the extent of the surround: smaller values of σa result in narrower surrounds.

Fig. 1.Flow process of the proposed method.

The normalizing factor is as follows:

The MSR reduces to SSR when β = 1, with the additional constraint that Wa = 1.

MSRCR introduced CR to produce good color renditions and also to preserve a reasonable degree of color constancy in the MSR result. The CR factor α is computed as follows:

We used MSRCR as the basic model of our image-quality improvement process [15]. The output of MSRCR is as follows:

2.2 Histogram matching

In the second step, we aim to make the image look natural. Thus, we convert the enhanced image to the logarithmic domain, and then map the log image through histogram specification. Our purpose is to find an appropriate shape for the specified histogram. Because a log-shaped histogram specification can render the mapped image bright enough, we represent the difference by increasing the pixels of gray levels, according to the intensity distribution of the input image. The specified histogram, p(s), is defined as follows:

where ε is a small constant and s is a positive integer within [0, 255].

To begin histogram matching, first, we need to find cumulative distribution functions (CDFs) of the log image at each color channel and our desired histogram. The CDF of an image is given as follows:

Similarly, the CDF of the specified histogram is defined as:

to find values of s that satisfy cdf(sn) = cIi(n), for n = 0, …, 255.

The value of sn is given by:

for n = 0, …, 255.

The resulting image can be obtained by the following transformation:

2.3 Edge-preserving smoothing

A bilateral filter is a nonlinear filter [24-25], where each pixel is replaced with a weighted mean intensity value of its neighborhood pixels, with the weights decreasing both with spatial distance and with a difference in value. The equation follows:

where f(x) is an input image; c(ξ,x) measures the geometric closeness between the neighborhood center x and a nearby point ξ; s(f(ξ), and f(x)) measures the photometric similarity between the pixel at the neighborhood center x and that of a nearby point ξ. Thus, the similarity function s operates in the range of the image function f, while the closeness function c operates in the domain of f. k1 is the normalized similarity function, for which the value is close to 1.

 

3. EXPERIMENTAL RESULTS

We compared our proposed method to SSR, MSR and MSRCR. We used images that lacked contrast over almost their whole area. Fig. 2 and Fig. 3 present the results from the different enhancement methods. Overall, all of the methods extracted contrast from the input images, providing much hidden information. However, SSR and MSR created an additional light source and a halo effect. These effects can lead to confusion regarding light direction, especially as shown in Fig. 2a. Moreover, SSR (Fig. 2b and Fig. 3b) led to an opaque effect, resulting in a very unclear image. SSR showed the worse results, changing some colors and creating a major halo effect along with a gray-world violation in some area of images. The above images are the original images under the condition with low intensity and normal or bright condition. We used the images in order to compare the result images when applying the enhancement process into the low intensity of original image. The bright condition image is used to see how differences the result images with the actual color of each object in the image

Fig. 2.The result of Image I: (a) Original Image, (b) SSR-ed image, (c) MSR-ed image, (d) MSRCR-ed image, (e) Proposed method image.

Fig. 3.The result of Image II: (a) Original Image, (b) SSR-ed image, (c) MSR-ed image, (d) MSRCR-ed image, (e) Table 1.Proposed method image.

MSRCR provided a better color distribution globally; the color of the yellow house is brighter than that in the other two images (Fig 2b and Fig. 2c). The original color of each object in the image becomes more apparent as the result of the color parameters used in that method, although it does not give an entirely satisfactory result. The stairs of yellow house seems unclear and cannot give the details of it.

In contrast, our method (Fig. 2e) reveals the bright yellow color of the house, and differences are also seen in the wall, which looks more natural than in the other images. Our method (Fig. 3e) efficiently increased both global and local contrast in the image. It also led to a more natural-looking scene even though some windows disappeared due to the balanced color distribution and the position of the light source. However, the hidden objects clearly revealed the houses of the behind the trees and the sky. These results indicate that our proposed method (Fig 2e and Fig. 3e) is superior to SSR (Fig 2b and Fig. 3b), MSR (Fig 2c and Fig. 3c), and MSRCR (Fig 2d and Fig. 3d).

Color colorfulness index (CCI) is used as the objective quality measurement for the result images. In [26], the image is assumed coded in the sRGB color space and the new colorfulness metric is given as:

where rg and yb are the components in sRGB color space. The higher CCI value the more colorful of the image.

In Table 1, our proposed method two sample images reach the next highest value in Image I and the highest value in Image II of CCI among the other methods in the experiment. It shows that it has the balance color in the result image, not too high or low in color distribution.

Table 1.CCI value of images

 

4. CONCLUSION

In this paper we have presented the ICT fusion technology developments which are several mathematic based the color image enhancements, image manipulations and image analyses. A novel color image-enhancement method was proposed. The method employs the retinex algorithm, followed by a histogram matching and filtering process. Experimental results showed that our method successfully enhanced image contrast, avoided halo effects, and maintained the color distribution in the image. Some local areas seemed less clear because a filter was used to smooth the images. This phenomenon is a drawback of our method; thus, other types of filters should be considered in future studies.