Priority-based Unequal Error Protection Scheme of Data partitioned H.264 video with Hierarchical QAM

Abstract

In this paper, we propose a priority-based unequal error protection scheme of data partitioned H.264/AVC video with hierarchical quadrature amplitude modulation. In order to map data with higher priority onto the most significant bits of QAM constellation points, a priority sorting method categorizes different data partitions according to the unequal importance factor of encoded video data in one group of pictures by evaluated the average distortion. Then we propose a hierarchical quadrature amplitude modulation arrangement with adaptive constellation distances, which takes into account the unequal importance of encoded video data and the channel status. Simulation results show that the proposed scheme improves the received video quality by about 2 dB in PSNR comparing with the state-of-the-art unequal error protection scheme, and outperforms EEP scheme by up to 5 dB when the average channel SNR is low.

1. Introduction

With advances in wireless networking technologies, there is a growing trend to access video services over portable multimedia devices. However, the varying characteristics of wireless channel, such as limited and time-varying bandwidth and error-prone, make video delivery over it a difficult and challenging task. Due to compressed video bitstream is extremely sensitive to transmission errors, a single error may result in obvious reconstructed video quality degradation of the current frame, but the effects may propagate to the successive frames. For example, because of the utilization of variable length coding used in H.264/AVC [1], a single bit error can cause the decoder to lose synchronization, resulting in the subsequent correctly received bits can become useless [2]. Therefore, to minimize the impact of the transmission errors on the reconstructed video quality, error resilient tools, such as data partitioning (DP) and flexible macroblock ordering (FMO), are utilized to improve the transmission robustness of video streams. On the other hand, unequal error protection (UEP) can be implemented by exploiting the characteristic of the unequally distributed significance of bits in the compressed video streams. In H.264/AVC, with data partitioning, each partition in one slice has unequal importance for reconstructed video quality due to their different dependency relationships. Because of the reconstructed video quality would be severely degraded when errors occur in these partitions with higher significance, they should be assigned more protection than those with less significance. So, the UEP scheme is expected to improve the reconstructed video quality.

Recently, many researches of hierarchical quadrature amplitude modulation (HQAM) applied to UEP scheme have been reported [3]-[10]. HQAM is a modification of QAM, which is a simple and efficient UEP scheme. In the HQAM, data with high priority (HP) are mapped onto the most significant bits (MSBs) of QAM constellation points, and data with low priority (LP) are mapped onto the corresponding least significant bits (LSBs). Furthermore, by adjusting the distance between constellation points, the bit error rate (BER) of HP data is improved at the cost that the BER of LP data is degraded. Therefore, the UEP of H.264/AVC video streams can work well at the physical layer via allocating unequal priorities to bits with different significance in the video streams[3]. Barmada et al. [4] first proposed an UEP scheme based on the HQAM for data partitioning of H.264/AVC video streams, and the performance of the proposed UEP scheme was evaluated over additive white Gaussian noise (AWGN) and Rayleigh fading channels. Shih et al [5] classified macroblocks into slice groups using the explicit mode of FMO to exploit the error resilient features of H.264/AVC, and proposed an UEP approach to protect effectively the streams. Chang et al. [6] took into account of the frame type and macroblock position in each frame to allocate priorities to bits in compressed video streams, the performance of the UEP scheme was also evaluated over the AWGN channel. In [7], the author proposed an UEP scheme based on the HQAM for H.264/AVC video transmission over frequency selective fading channels, in which an OFDM subcarrier classification strategy with two signal-to-noise ratio (SNR) thresholds was presented. In [8], a combination of turbo code and HQAM was used to provide unequal error protection for two-layer scalable H.264 bitstream. In these schemes, HQAM is used to provide two-level priority: HP and LP, which are suitable for layered scalable video encoders such as H.264/AVC. Although the UEP schemes based on HQAM in literature can provide better error resilience for the compressed video streams over wireless channel, the significance factor of NAL units in H.264/AVC is not quantified, and the frequency-selective fading characteristic of the wireless channel, have not yet been considered. In [9], the coded bits of each frame were hierarchically partitioned into HP and LP substream, using two-staged partitioning strategy. Then pairs of stream blocks, one from each substream were selected judiciously to form symbols, which were modulated using HQAM with varying modulation parameter. The author proposed an unequal error protection (UEP) scheme based on HQAM for 3-D video transmission [10]. Considering the color sequence has a more significant impact on the reconstructed video quality, the color data with high priority were mapped onto the most significant bits of the 16-QAM constellation points and the depth map with low priority were mapped onto the less significant bits.

In this paper, we propose an efficiently-mapped HQAM based on priority of data partitioned H.264 video stream transmission over wireless channel. Data partitioning in H.264/AVC is used to divide the video streams into substreams with different priorities. We present a priority sorting method to evaluate the importance of different data partitions by calculating distortion in one group of pictures (GOP). Then taking into account the unequal importance of encoded video data, we provide a two-level HQAM arrangement with adaptive constellation distances to improve the reconstructed video quality.

The rest of this paper is organized as follows. In Section 2, the fundamental of the hierarchical QAM and data partitioning in H.264/AVC standard are overviewed. In Section 3, we calculate the importance factor for each data partition in one GOP, and each data partition is marked with high or low priority. Section 4 describes the proposed scheme using efficiently mapped HQAM. Section 5 includes simulation results and discussions. Finally, the paper is concluded in Section 6.

 

2. BACKGOUNDS

2.1 Fundamental of the hierarchical QAM

Modulation is one of the key functions performed at the physical layer in transporting information over wireless networks. It is desirable to choose higher order modulation scheme due to high bit rate requirements of applications such as video. The higher order modulation schemes increase the transmission capacity by assigning more bits to each transmitted symbol. However, modulation schemes that allow a larger number of bits per symbol have symbols closer to each other in the constellation diagram, and small errors can result in erroneous decoding. To address on this problem, the use of non-uniform signal space constellation is suggested to give different degrees of error protection[11]. The basic philosophy of using hierarchical modulation as an alternative to FEC-based UEP is that important information is protected without any additional bandwidth requirement[12].

The hierarchical QAM was initially proposed to provide different classes of data, such as video or text files, to the end users in different wireless reception conditions. In hierarchical QAM, the signal constellation is partitioned so that specific blocks of the partition contain message points with the maximum possible Euclidean distance between them at the expense of message points in other blocks which are separated by a much smaller minimum distance[13]. The coding of the message points is chosen so that the specific bits assigned to the widely separated points have lower error probabilities than the others[14]. The data stream can be split into two separate substreams so that the most important bits are assigned higher priority and are known as HP bitstream. The remaining bits are assigned lower priority and are known as LP bitstream.

The conventional HQAM with signal constellation size M offers two levels of priority. HP data occupies the first two MSBs of each point. The Gray coded hierarchical 16-QAM constellation map is depicted in Fig. 1, where d1 is the minimum distance between constellation points in different quarters, and d2 is the minimum distance between constellation points in the same quarter. The hierarchical QAM modulation parameter α is defined as α = d1/d2 . Gray code labeling permits all points belonging to the same quadrant to have the same HP bits. This means that if the received point is demapped erroneously to a neighboring point but remains within its constellation quadrant, the HP bits will remain uncorrupted. LP data occupies the rest of the bits in the point label. For M-point constellation, the number of LP bits in each symbol is given by .

Fig. 1.Two-level constellation diagram for 16-HQAM

For the case α = 1, the constellation map in Fig. 1 is reduced to a nonhierarchical 16-QAM with the same reliability for each bit in QAM symbols. If α > 1, the signal constellation becomes that of an HQAM. By controlling the value of α , it is possible to control the bit error (BER) of the HP and LP bitstreams. For a given average signal power, MSBs of QAM symbols on which HP data are mapped could gain more protection than LSBs on which LP data are mapped by adjusting d1 and d2 to make α > 1 . Furthermore, increasing the value of α will increases the HP protection but decreases the LP protection.

In the AWGN channel, the BER performance of the 16-HQAM as a function of channel carrier-to-noise ratio. BER of MSBs of 16-HQAM symbols can be calculated as [3]

where α is the HQAM modulation parameter, γ is the average signal-to-noise (SNR) per bit and γ = Eb / N0 , erfc(x) is the complementary error function which can be expressed as .

BER of LSBs of 16-HQAM symbols is

To simplify Equation (2), the complementary error function erfc(x) can be approximated as [7]:

By substituting Equation (3) into Equation (1), the BER of MSBs symbol in AWGN channel can be expressed as

Le X = exp(-γ), then Equation (4) can be rewritten as

Taking the logarithm of both sides in Equation (5), we can obtain

Let , then we have

By solving Equation (7), select one solution of α greater than 1 as the HQAM modulation parameter. From Equation (7), given bit signal to noise ratio γ and PMSB , we can calculate the modulation parameter α . By substituting the modulation parameter α and bit signal to noise ratio γ to Equation (3), we can obtain the BER of LSBs symbol PLSB .

Fig. 2 illustrates the BER performance of HP and LP data in the 16-HQAM over the AWGN channel with different values of α . As we can see, when α > 1, the HP data would gain more protection than the LP data, and by increasing the value of α , the BER performance of the HP data is improved while that of the LP data is degraded. When α < 1, the LP data would gain more protection than the HP data, with the consequence that the BER of LP data is lower than that of HP data. When α = 1, the hierarchical QAM is reduced to a nonhierarchical QAM with the same reliability for each bit in QAM symbols. Especially when Eb / N0 = 10dB and α = 1.6, the BER difference between HP data and LP data gains three orders of magnitude.

Fig. 2.BER performance of HP & LP data in 16-HQAM with different α

2.2. Data partitioning in H.264

Data partitioning is an effective application-level framing technique in H.264/AVC standard, which divides the compressed video data into separate units of different importance. With data partitioning, each partition in one slice has unequal importance for the reconstructed video quality due to their different dependency relationships. Normally, all symbols of Macro Blocks (MBs) are coded together in a single bit string that forms a slice. However, data partitioning creates more than one bit string (partition) per slice, and allocates all symbols of a slice into an individual partition with a close semantic relationship. There are three different partition types in H.264: 1) partition-A, contains the header information which is composed of MB types, quantization parameters (QPs), and Motion Vectors (MVs). This information is the most important because without partition-A, symbols of the other partitions are no longer useful during decoding. 2) partition-B, also called the intra partition, is composed of intra coded block patterns (CBPs) and intra coefficients. Partition-B requires the availability of partition-A to be useful at the decoding level. In contrast to the inter information partition, intra information can stop further drift and hence is more important than the inter partition. 3) partition-C, also called inter partition, contains inter CBPs and inter coefficients. Since intra coding can stop error propagation, partition-B is usually considered more important than partition-C.

Usually, if partition-B or partition-C are missing, the available header information can still be used to improve the efficiency of error concealment. More specifically, due to the availability of the MB types and motion vectors, a comparatively high reproduction quality can be achieved as only texture information is missing.

 

3. Calculating the importance factor

As we mentioned above, partition-A can be decoded interdependently, partition-B and partition-C depend on partition-A for decoding. The importance of different partition in one GOP is different, because of the extent of error propagation caused by transmission errors depends on the position of the error in the coded video sequence and the relationship between different data partitions. For example, transmission errors in a P-frame immediately right after an I-frame will affect all the following frames in the GOP whereas an error in the last P-frame in the GOP does not affect any other frames. It is therefore beneficial to quantify the influence to the video quality of each partition, and the quantifying algorithm for evaluating the importance of different data partitions is described as follows.

In one GOP, which consists of one intraframe, i.e. I-frame or instantaneous decoding refresh (IDR) frame, and a set of interframes, i.e. P-frames and B-frames. The intraframe is the predictive reference frame of subsequent interframes, so it is more important than the interframes. Furthermore, previous P-frames are more important than succeeding P-frames, and B-frames are the least important in the event that hierarchical B-frames are not applied. Therefore, the video transmission schemes provide unequal error protection to frames with different encoding types, and the importance measurement is prerequisite for designing an effective UEP scheme. In this section, we describe the importance evaluating method by calculating the distortion.

As we mentioned before, a GOP is composed of one IDR frame followed by L-1 P-frames, where L denotes the total number of frames in one GOP. The IDR access unit contains information that cannot be partitioned into the three partitions. That is, they contain only the intra picture (coded picture that can be decoded without needing information from previous pictures) where no data partitioning can be applied. Thus, the generated slices (partition-A, partition-B, partition-C, PSC, and IDR) are directed to the network abstraction layer (NAL) with an associated header using an implementation dependent data structure. In this paper, due to its importance, the IDR frame is assumed to be perfectly protected and correctly received, we just considering about the following L-1 P-frames.

Now, we define some notations used in the derivation of the priority model. Let Nf be the pixel number of a frame, let F(i,m) be the original value of m-th pixel in i-th frame, let Fc(i,m) and Fd(i,m) be the reconstructed value at the encoder and decoder respectively. The source distortion Ds(i), channel distortion Dc(i) and total distortion D(i) of i-th frame are calculated under Mean Square Error (MSE) criterion as:

The relationship of Ds(i), Dc(i) and D(i) is:

To analyze the importance of different partitions in i-th frame, we derive the distortion of each partition in i-th frame under the assumption that only the partition under discussion is lost and other partitions in i-th frame and all partitions in subsequent frames are error-free. To evaluate the influence factor of different partition, the error-concealment methods are adopted [15]:

1) in case of partition-A of i-th frame is lost, the whole frame is corrupted, because partition-B and partition-C are dependent on it. Then for all pixels in the current frame, the k-th reconstructed pixel in i-th frame can be calculated as

2) in case of partition-B of i-th frame is lost, the pixels in an intracoded MB are affected, but the reconstructed pixels in an intercoded MB can be obtained by motion compensated prediction with the correct motion vector, reference frame, and residual data. Let denotes the pixel from which fik is predicted, and refers to the quantized prediction error. Then the reconstructed pixel can be formulated as

3) in case of partition-C of i-th frame is lost, the reconstructed pixel in an intracoded MB will not be influenced when constrained intraprediction is utilized. Then the reconstructed pixel can be formulated as

In case of partition-A, partition-B or partition-C in i-th frame is lost, it will influence decoding i-th frame and the following frames. Let D(i, j, k) denotes the distortion of j-th frame caused by the lost of k-th partition of i-th frame, 1 ≤ i ≤ L - 1,0 ≤ j ≤ L - 1, and k = {0,1,2}. k=0 denotes as partition-A, k=1 is partition-B and k=2 is partition-C. To quantify the influence of each data partition of i-th frame upon the recovery quality within one GOP, we define the distortion change ΔD(i, k) as the distortion difference between two cases: k-th partition of i-th frame is lost and received without error, and the average distortion change within one GOP is defined as the recovery quality metric or the importance factor.

Then is calculated as:

where .

Substituting D(i, j, k) and Equation (11) into Equation (12), we can represent as:

Let be the average distortion changes of the intra and inter-MBs respectively caused by k-th partition lost, then Equation (13) can be represented as:

As mentioned above, the lost of partition-A in i-th frame will not influence i-th frame decoding, but also the following frames after i-th frame within one GOP. Then we recursively calculate the propagated influence of the following frames, from (i+1)-th frame to L-th frame within one GOP as:

Where Fd(i + 1,m) denotes the predictive value of m-th pixel in (i+1)-th frame, denotes the corresponding difference information of m-th pixel. From Equation (15) and (16), using Equation (14), we can calculate the average distortion change within one GOP caused by partition-A, partition-B or partition-C loss respectively.

Based on the analysis of for frames from 2 to frame L-1, a priority sorting method jointly taking account into the unequal importance of partitions in one frame and P-frames at different positions in one GOP is presented. That is, IDR frame, PSC and all partitions of the first m P-frames and partition-A of the middle n P-frames are labeled as HP, and partition-B and partition-C of the n P-frames and all partitions of the remaining p P-frames in one GOP are labeled according to the calculated average distortion change as:

 

4. Proposed UEP scheme with importance factor

As we mentioned above, 16-HQAM can offer two-level of priority. In Section 3, the importance factors of data in H.264/AVC video stream are calculated and the coded video data can be sorted into HP and LP data. In this section, we describe the proposed UEP scheme based on importance factor. The system model of the proposed HQAM-based UEP scheme for H.264/AVC video stream is depicted in Fig. 3. With data partitioning, the H.264/AVC video streams are encoded and sorted into two substreams with unequal priorities: HP and LP substreams. HP substream consists of bits that are highly sensitive to noise and a single bit error may cause the loss of synchronization between the encoder and decoder and may cause the failure of the reconstruction process after it occurs. LP substream consists of bits which have less severe effects of errors.

Fig. 3.System model diagram of the proposed UEP scheme

Due to two-level HQAM with fixed value of α for UEP suffering from the problem that α increases, protection to HP bits increases, but at the cost of LP bits, thereby reducing its performance at higher carrier-to-noise ratio. Further, the use of a constant value of α assigns the same level of protection to all HP bits relative to LP bits. In our UEP scheme of 16-HQAM, if the average SNR is low (i.e. SNR≤20dB), symbols are formed by combining two MSB bits from HP substream and two LSB bits from LP substream, symbols can be modulated with large value of α (say α1). If the average SNR is high (i.e. SNR>20dB), symbols can be modulated with smaller value of α (say α2 < α1). The scheme is expected to perform better than HQAM as the value of modulation parameter is changed according to the channel status.

 

5. Simulation results and analysis

In this section, the performance of the proposed UEP scheme is presented. In the simulations, the video codec is implemented based on H.264/AVC reference software JM15.1 [16], and three video test sequences Foreman, Susie and Coastguard in Quarter Common Intermediate Format (QCIF, 176×144) are used. All sequences are encoded 100 frames with the coding structure of IPPPPP at 30f/s. Error concealment is applied to the decoded frames as mentioned in Section 3. The hierarchical QAM and OFDM system with 256 subcarriers are implemented in MALAB with the assumption that the available channel rate of the OFDM system is 100kb/s. The SUI-3 wireless channel model [17] is chosen as the frequency-selective fading channel model. The channel condition is considered constant during the transmission of one video frame. The modulation parameter α1 and α2 are set to 1.4 and 1.6, respectively.

Simulations with different values of the average SNR per bit γ are carried out 30 times to obtain reliable simulation results. The average PSNR (Peak Signal-to-Noise-Ratio) is used to evaluate the performance of the proposed UEP scheme. Comparing with the EEP and UEP scheme in [6], the average PSNR performance of the proposed UEP scheme is shown in Fig. 4-6. As we can see, the proposed UEP scheme outperforms EEP by up to 5dB at a low average channel SNR ( Eb / N0 = 19dB ), which means that the reconstructed video quality of the proposed UEP scheme is better than that of EEP. When compared with the UEP scheme in [6], the proposed UEP scheme can also provide up to 1.3~4.8dB gain when the average channel SNR is low. That is because in the low average channel SNR, due to frequency-selective nature of the wireless channel, most of the available subcarriers are in deep fading, then EEP and UEP scheme in [6] cannot provide appropriate protection for the HP data mapped onto subcarrier in deep fading. However, the proposed UEP scheme can avoid mapping HP data onto subcarrier in deep fading, resulting in the significant improvement of the reconstructed video quality at the receiver. The performance of the three simulated schemes are improved with the increasing of the channel SNR, but the proposed UEP scheme still outperforms the EEP and UEP scheme in [6]. When the average channel SNR is high enough (e.g. Eb / N0 ≥ 26dB ), the channel status is good. In such case, the proposed scheme could maintain a high average PSNR performance as well as the other two simulated schemes.

Fig. 4.Average PSNR of Foreman video sequence

Fig. 5.Average PSNR of Susie video sequence

Fig. 6.Average PSNR of Coastguard video sequence

To further illustrate the performance of the proposed UEP scheme at low average SNR (i.e. Eb / N0 = 11dB ), the average PSNR performance with different α is shown in Fig. 7. As we can see, when the wireless channel is in bad status, the average PSNR is increasing with the modulation parameter α .

Fig. 7.Average PSNR at Eb / N0 = 11dB with different α

 

6. Conclusions

A hierarchical QAM-based UEP scheme for H.264/AVC video streams over wireless networks is proposed in this paper. Data partitioning is used to divide the video stream into substreams with different priorities. The novelty of the proposed scheme is based on priority sorting method that categorizes different data partitions according to the unequal importance of encoded video data in one group of pictures by evaluated the average distortion. Then substreams with different priorities are unequally protected by using subcarrier mapping and the hierarchical QAM modulation. Simulation results show that the proposed UEP scheme outperforms the EEP and the existing hierarchical QAM-based UEP schemes, especially when the average channel SNR is low.