Video Coding Technique with Multi Objective Particle Swarm Optimization and EZW

1. Introduction

The storage and transmission of the video signals are not possible without compression, since video information of one second requires several megabytes of memory. Also transmission of video signals, need the use of compression techniques to reduce the transmission bandwidth. A lot of research works on compression algorithms have been carried out for video coding. At high compression ratios, block based coders introduce artifact and ringing effects. While considering the wavelet based motion compensated 2D+t transform for video coding, the complex motion estimation and compensation is a tedious process. So 3D-Wavelet based coders are proposed. Wavelet based motion compensated video coders are recently developed. 3D temporal wavelet transforms for video compression is reported [1, 2] and is shown that 3D wavelet transforms without motion compensation provides better compression performance than the motion compensated predictive methods. The popular encoding techniques such as EZW [3] and SPIHT [4] combined with 3D wavelet results, good quality of compression. These coders provide both temporal and spatial scalability [5]. Continuous rate scalable applications can prove valuable in scenarios where the channel is unable to provide a constant bandwidth. Rather than terminating the session, a decoder can adjust the bit rate to use the limited resources, yet produce video of acceptable quality. Such decoders are particularly attractive because of their flexibility. Scalable video coding has the capability of reconstructing lower resolution signals from partial bit streams. Motion compensated temporal wavelet coding eliminates the encoder drift in scalable video coding [6]. The limitations of discrete wavelet transform for multi dimensional signals such as aliasing, oscillatory nature of coefficients and lack of directionality are reported [7]. Dual Tree Discrete wavelet (DDWT) Transform is more suitable for video coding with the kernel functions having the capability of directional property, eliminating motion estimation and compensation process. The suitability of Dual tree Discrete Wavelet Transform for video coding is already discussed [8-9]. The DDWT filter bank design and its application for denoising is also discussed [10].

Dual tree discrete wavelet transform is an expansive type transform, since it converts M number of samples into N number of coefficients (M

Optimizing the problem with single objective often leads to unacceptable results with respect to the other objectives. Therefore a perfect multi objective solution that simultaneously optimizes each objective function is almost impossible. One solution for this multi objective problem is to find a set of solutions which satisfies all the objectives in an acceptable level and at the same time it does not dominate by any other solution. In video coding, the computation time, compression ratio and the quality of the decompressed video are important constrains. If we go for high compression ratio, the quality of the reconstructed video will be is low. For video transmission, the compression ratio and the computation time are important factors.

Here in this paper, the multi objective PSO is used to find the set of optimum wavelet coefficients, which satisfies the more than one of the above mentioned constraints.

In paper [14], video coding is formulated as a multi objective problem and the objective functions are identified as MSE, Coefficient reduction factor. The weighted aggregate method of MOPSO is used to find the optimum subbands of DDWT. Next MOPSO with weighted aggregate method, is analyzed by varying the number of particles and threshold values [15]. The Vector evaluated MOPSO method is compared with the weighted aggregate method and their performances are measured in paper [16].

In this proposed work, the suitability of EZW for encoding the selected subbands using MOPSO is analyzed. The video sequence is decomposed by 2-D Dualtree discrete wavelet transform. Noise shaping scheme is used to Select the significant coefficients from the vast DDWT coefficients. By using the multi objective PSO algorithm, the Dualtree subband coefficients with high energy content are identified. The objective functions are considered as MSE, Computation time and Entropy. The identified subband coefficients in each plane are Encoded using EZW algorithm. The paper is organized as follows.: Section II explains the PSO and Multi Objective PSO. Video coding process using multi objective PSO is explained in section III, while section IV presents the experimental results. Conclusion is summarized in section V.

1.1 Particle swarm optimization

Particle Swarm Optimization (PSO) is a global optimization technique based on swarm intelligence. It is inspired from bird flocking [17]. It is widely accepted and focused by researchers due to its profound intelligence and simple structure. Currently PSO has been implemented in a wide range of research areas such as functional optimization, pattern recognition, neural network training and fuzzy system control etc., and is successful. In PSO, each potential solution is considered as one particle. The system is initialized with a population of random solutions (particles) and searches for optima (global best particle), according to some fitness function, by updating particles over generations; that is, particles “fly” through the N-dimensional problem search space to find the best solution by following the current better-performing particle. The variants of PSO’s such as Binary PSO, Hybrid PSO, Adaptive PSO and Dissipative PSO are used in various image processing applications.

For binary PSO [18], the position of the particles are represented in binary space. Hybrid PSO combines the basic mechanism of PSO and the natural selection mechanism, which is usually utilized by Evolutionary Computation methods such as GAs.

Around 10% of PSO applications are devoted to image and video analysis applications. Image analysis applications include iris recognition, fruit quality grading, face detection and recognition, image segmentation, synthetic aperture radar imaging, Image fusion and Image registration etc., Video application includes MPEG optimization, motion estimation, object tracking, body posture tracking and traffic incident detection.

In [19], sparse signal recovery using PSO for Image compression is discussed. Images are compressed with compressive sampling and then reconstructed with PSO techniques. Several enhancements to the basic PSO algorithm are discussed to improve signal recovery accuracy. Fractal image compression is one of the image compression techniques in the spatial domain but it involves more computational time due toglobal search. In order to reduce the computational time and acceptable quality of the decoded image, PSO algorithm is proposed [20].

Recently PSO has been extended to deal with multiple objective optimization problems[21]. The fixed population size MOPSO and variable population size PSO (Dynamic PSO) are used throughout the evolution

In PSO, we assume that the problem is in a D-dimensional space, which includes many particles; each particle represents a feasible solution of optimization problem. In every iteration, each particle updates itself by the two extreme values, one is individual extreme value pbest, which is personal best value for that particle pid , and the other is the global best value for that particle (gbest) pgd .

Each particle adjusts its flight speed and direction according to current rate, pbest and gbest using Eq. (1) and Eq. (2) repeatedly.

Where N is the number of particles and D is the dimensionality; Vi =( vi1 , vi2 ,…, viD ),

vid ∈ [− vmin , vmax] is the velocity vector of particle i, which decides the particle’s displacement in each iteration.

Similarly, Xi = ( xi1 , xi2 , …, xiD ),

xid ∈ [− xmin , xmax] is the position vector of particle i which is a potential solution in the solution space.

The quality of the solution is measured by a fitness function; w is the inertia weight which decreases linearly during a run; c1, c2 are both positive constants, called the acceleration factors which are generally set to 2.0; r1(.) and r2(.) are two independent random number distributed uniformly over the range [0, 1]; and pgd, pid are the best solutions discovered so far by the particle in the group and itself respectively.

vid (t+1) -velocity of the particle at time (t+1) xid (t+1) -position of the particle at (t+1)

In Eq. (1), the first part is momentum which is effect of the current state of the particle; the second part is the individual cognitive part which adjusts particle’s flight to its own positions; the third part is the community part which guides the particle’s flight to the group’s best position. The balance between the three parts determines the PSO’s searching ability.

PSO as developed by the authors comprises a very simple concept, and it can be implemented in a few lines of computer code. It requires only primitive mathematical operators and is computationally inexpensive in terms of both memory requirements and speed.

1.2. Basic concepts of multi objective optimization

Most of the problems in the real world are multi objective in nature and may have multiple optimum solutions for different objectives. The general Multi-Objective Optimization Problem is defined [22].

Def. 1: Find the vector (Solution)

which satisfies the p inequality constraints:

and the q equality constraints:

and optimizes the vector function:

The constraints given by Eqs. (4) and (5) define the feasible region Ω (solution region) and any point in Ω defines a feasible solution. The l components of the vector are the criteria to be considered. The constrains and represent the restrictions imposed on the decision variables. The vector denotes the optimum solution. When there are several objective functions, the concept of optimum changes lead to “trade-off” solutions rather than a single solution.

1.2 Multi objective PSO

Solving Multi objective optimization problems using PSO is discussed [23]. PSO is particularly suitable for multi objective optimization because of high speed of convergence. The usage of Genetic Algorithm for multi objective problems is reported [24]. PSO is tried for optimization using two approaches (single objective and multi objective) in paper [25]. The multi objective PSO is used for Image segmentation [26] and filter bank design [27] optimization problems. In order to handle multiple objectives, PSO must be modified before being applied to MO problems. In most approaches, the major modifications involve the selection process of global best and personal best. A grid based global best selection process is employed a second population to store the non dominated solutions [28]. From the second population, using Roulette wheel selection, the global best is selected randomly. The personal best is selected according to the Pareto dominance.

The average PSNR values of two video sequences Foreman and Rhinos using different coding techniques are tabulated in Table 1.

Table 1.Average PSNR comparison of various video coding techniques

In DDWT+PSO Technique, Single objective function that is maximizing the PSNR only considered. But in Multi objective PSO, factors like coefficient reduction factor (related to compression ratio), PSNR(related to minimum MSE) are considered. The MOPSO optimally performs the compression with good quality of the reproduced video. The PSNR values are shown in the Table 1.

In the proposed work, the weighted aggregate approach is used to find the optimal subbands of the DDWT. The fitness function of the MOPSO combines the MSE, ESUM-Entropy of the various subbands and Computation time as objective functions with different weightage values. The EZW algorithm is used to encode the selected coefficients using MOSPO.

 

2. Video Coding using Multi Objective PSO

At the expense of less number of bits, the video sequence may be transmitted and reconstructed with good quality. The input video sequence is subjected to three level decomposition using 2D Dual tree Discrete Wavelet Transform (DDWT). The DDWT coefficients are reduced to desired numbers using noise shaping process. The coefficients after noise shaping are subjected to MOPSO block, where the highest energy content coefficients are selected based on objective functions. The selected subbands are encoded using EZW algorithm and the parameters PSNR,Computation time are measured for different frame rates and bits/pixel values. The block diagram of the proposed system is shown in Fig. 1.

Fig. 1.Block diagram of the proposed video coding system

2.1 Noise shaping

Noise shaping is an iterative projection based algorithm. The dual tree wavelet coefficients are subjected to this Noise Shaping Scheme in order to select the significant coefficients from vast DDWT coefficients. It is faster when comparing with the matching pursuit algorithm.

More than one coefficient is selected in iteration. The simple threshold (θi) operation leads to loss of information. The information loss is compensated in this iterative projection based noise shaping. After performing simple threshold operation, the retained coefficient magnitudes are modified by multiplying them by a small constant s. This is done for the compensation of the loss of small coefficients due to threshold operation as illustrated in Fig. 2, the coefficients are obtained by running the iterative projection algorithm with a preset initial threshold, and gradually reducing it until the number of remaining coefficients reaches R, a target number. In each iteration, the error coefficients are multiplied by a positive real number s and added back to the previously chosen large coefficients, to compensate for the loss of small coefficients due to threshold operation. Noise Shaping requires substantially fewer computations, to yield the set of coefficients that can yield the same representation accuracy.

Fig. 2.The noise shaping scheme

2.2. MOPSO problem formulation

Fig. 3 shows the video coding multi objective problem formulation. The objective functions are Mean Square Error (MSE), Computation Time and Entropy of the subbands (ESUM). All the three are minimization functions. The fixed population size MOPSO is used throughout the evolution process to explore the search space to discover.

Fig. 3.MOPSO block in the proposed system

The non dominated individuals (particles). Here the constraints are frame rate and bits per pixel. The bits/pixel values are taken as 0.3, 0.5 and 0.8. A frame rate of 5 Frames/sec is taken for analysis.

2.3. Optimal subband selection using MOPSO

Minimum entropy principle is adapted here to find the best wavelet coefficient sets. The sum of all the entropy values of the node (ESUM) of the selected coefficients in all subbands are calculated. Fitness function for PSO is defined by considering MSE, ESUM and Computation Time. In PSO process, the fitness function values are in descending order and the minimum is the global optimum value. MSE and ESUM as objective functions for selecting the best wavelet packet [29]. In the proposed work, the fitness function is defined as follows with three objective functions:

The constraints considered are 0.3 or 0.5 0r 0.8 bits/pixel and a frame rate of 5 Frames/sec. Where α1, α2, α3 are constants and their (weightage values) values are taken as 0.4 and 0.3 and 0.3 respectively.

The Entropy sum (ESUM) is calculated for each set coefficients in the subbands. Example as Entropy of (LL, LH, HL and HH).

The computation time varies with respect to the size of the subband sets. It includes the entire optimum coefficient selection using MOPSO and EZW encoding time. The average Mean Square Error (MSE) σε is calculated as per equation

Iorg (i, j, k) - Original Image frame in the video sequence Ireco (i, j, k) - Reconstructed frame in the video sequence

Where K is the number of frames in the video sequence.

 

3. Optimal subband selection Process

Step 1: Divide the video sequence into group of Frames.

Apply 2D Dual tree wavelet transform and subject the coefficients to noise shaping algorithm. The result is a matrix with the reduced number of Coefficients, which is used for particle generation.

Step 2: Initialization of population: Set of randomly selected subband coefficients are considered as Particles. Each particle is initialized as the multiplication of randomly initialized matrix with subband coefficient matrix. The randomly initialized matrix is consisting of ‘0’s and ‘1’s distributed randomly.

Step 3: Apply inverse dual tree wavelet transform. The reconstructed image is obtained. Calculate MSE, PSNR and Computation time values.

Step 4: Calculate each particle’s fitness value according to the Eq. (7).

Step 5: If the particle’s fitness value is better than the particle’s best fitness value, then Pid (individual best) is updated. If the fitness value is better than the global best fitness value, then Pgd (global best) is updated.

The particle’s velocity is the probability of a particular subband to be included in the set of subbands. The position updating is the inclusion or deletion of a particular coefficient in the particle’s set of coefficients. The particles velocity and position updating are determined based on the global best particle. The set of coefficients in the particles are modified according to the coefficients in the global best particle.

Step 6: Continue the exploration process until a pre specified iterations are satisfied. Declare the global optimum value as the solution (Optimum set of coefficients).

For the given constraints in terms of bits/pixel and frame rate, the ESUM, MSE and the Computation time are measured. According to the weighted aggregate approach the best particle – set of optimum subband coefficients is selected. The selected coefficients in the subbands are encoded using EZW algorithm. The average value of PSNR and is calculated for different bit rates.

The particle updating parameters are taken as the inertia weight w =0.9 (fixed value) for each iteration and c1=2, c2=2,

The number of particles used: 30, PSO Search: Number of iterations: 10 Threshold for Noise shaping: 128 Number of Noise shaped coefficients: 16,000. EZW no. of passes: 10 At level 3, decomposition, the entropy values are calculated as Entropy Sum(LL) = 0, Entropy Sum(LH) = 1.3240 Entropy Sum(HL) = 1.0974 Entropy Sum(HH) = 1.4850 PSNR value of the sequence is calculated as

where Imax =255, and σε -Mean Square Error.

The PSNR is calculated with MOPSO for different frame and bit rates using Eq. (9).

 

4. Results

The video sequence is first grouped into 5 frames and is subjected to DDWT decomposition. The filter bank used for DDWT is as discussed [10]. After DDWT decomposition the number of coefficients is minimized using Noise Shaping algorithm. The target number of coefficients for noise shaping is fixed as 1,16,000 and the multiplication factor value s =1.8 is assigned to compensate the loss of coefficients due to thresolding during the process. The initial threshold for noise shaping is fixed as 128. The threshold value is decreased to zero by a step 1 for the remaining iterations. Then the MOPSO optimizer selects the optimal subbands with the given constrains. MOPSO further reduces the number of coefficients. The high energy content coefficients are selected. The selected coefficients are encoded using EZW method. Dualtree discrete wavelet transform has two trees. Totally 20 subbands are obtained after the three level decomposition. The subbands of Foreman frame 1 is shown in Fig. 4. The twenty subband coefficients average value in the respective subbands are encoded using EZW algorithm. The standard video sequences Foreman, Rhinos and Akiyo are used to test the performance of the proposed method.

Fig. 4.3-level DDWT Decomposition of Foreman frame number 1 with 20 subbands.

For EZW encoding the maximum coefficient value for threshold calculation is taken as 1024. The total number of passes in EZW coding is 10. At the end of the 10th pass the threshold value is reduced as 2.The EZW scanning order of wavelet coefficients are shown in figure 5. The subband structure and Encoding are same as like DWT + EZW encoding. The difference is DDWT contains more subbands than DWT, since it is an expansive transform.

Fig. 5.Scanning order of wavelet coefficients in different subbands and EZW coding process

The average PSNR value of video sequences under various methods such as DWT+3D SPIHT, DDWT and MOPSO with DDWT and MOPSO with EZW encoding are given in Table 2.The results are compared with the standard 3-D SPIHT algorithm. From the Table 2, it is concluded that for a video sequence which has many edges and motions, such as Foreman sequence, DDWT with MOPSO+EZW outperforms SPIHT by more than 1 dB. Similarly for the other two sequences its performance is better than the SPIHT.

Table 2.Average PSNR performance comparison of the proposed technique

The performance measurements for the Foreman sequence for various bit rates are given in Table 3. The increase in number of bit/pixel results in considerable increase in PSNR value for all the techniques.

Table 3.Average PSNR comparison at different bits per pixel of the proposed technique with other techniques(Foreman sequence)

The Proposed MOPSO’s PSNR performance for the compression ratio of 0.8 bpp is around 1.2 dB (Foreman sequence) greater than the 3D SPIHT and around 4 dB greater than the DDWT (without MOPSO) techniques. Table 4 shows the computation time comparison of the proposed method with existing methods. The computation time (Searching time for the best particle) is around 40 to 45 seconds. It is based on the selected set of DWT subband coefficients.

Table 4.Computation time comparison

Fig. 6 shows the variation PSNR value for different frames of Foreman sequence. The PSNR value is higher than the SPIHT and also the variation is less in case of DDWT + MOPSO. There is a very variation in PSNR values with increasing frame rates.

Fig.6.Average PSNR value plot of foreman sequence

Table 4 Shows the computation time of the DWT+ SPIHT, DDWT+EZW and DDWT+MOPSO WITH EZW techniques for various video sequences. The computation time of the proposed technique (DDWT+MOPSO with EZW) is 100 seconds less when compared to the 3D SPIHT coding. The results of EZW encoding are more suitable for the output stage in the proposed system. The original and reconstructed video frames of Foreman and Rhinos sequences are shown in Fig. 7 and Fig. 8 respectively.

Fig. 7.Original frame −20 in Rhinos sequence and its minimum error reconstructed Frame using MOPSO Technique

Fig. 8.Original frame −52 in Foreman sequence and its minimum error reconstructed frame using MOPSO Technique

 

5. Conclusion

In this paper, a video coding technique using Dual tree Discrete wavelet transform combined with weighted aggregate MOPSO method with EZW is proposed. The performance of the proposed method is tested on test video sequences. The test results are compared with the standard DWT + SPIHT coding method. The obtained results are better than the existing standard technique. DDWT+ MOPSO with EZW encoding technique provides better PNSR value with less computation time. In future, the variants of the PSO and also the other approaches for Multi objective optimization will be considered to improve the performance of the system and reduce the computational complexity in MOPSO computation part.