Evaluation of Machine Learning Methods to Reduce Stripe Artifacts in the Phase Contrast Image due to Line-Integration Process

선적분에 의한 위상차 영상의 줄무늬 아티팩트 감소를 위한 기계학습법에 대한 평가

Abstract

The grating interferometer provides the differential phase contrast image of an phase object due to refraction of the wavefront by the object, and it needs to be converted to the phase contrast image. The line-integration process to obtain the phase contrast image from a differential phase contrast image accumulates noise and generate stripe artifacts. The stripe artifacts have noise and distortion increases to the integration direction in the line-integrated phase contrast image. In this study, we have configured and compared several machine learning methods to reduce the artifacts. The machine learning methods have been applied to simulated numerical phantoms as well as experimental data from the X-ray and neutron grating interferometer for comparison. As a result, the combination of the wavelet preprocessing and machine learning method (WCNN) has shown to be the most effective.

격자간섭계는 한 위상 물체에 의한 파두의 굴절변화로 인해 그 물체에 대한 미분 위상 영상을 제공하며, 이 미분 위상 영상은 위상 영상으로 전환되어야 할 필요가 있다. 미분 위상차 영상으로부터 위상차 영상을 얻기 위한 선적분 과정은 노이즈를 축적하고 줄무늬 아티팩트를 생성한다. 줄무늬 아티팩트는 선적분이 수행된 위상차 영상에서 적분 방향으로 노이즈와 왜곡이 증가한다. 이 연구에서는 이러한 아티팩트를 줄이기 위해 몇 가지 기계 학습 방법들을 구성하고 비교하였다. 기계 학습 방법들은 상호비교를 위하여 시뮬레이션 된 수치 팬텀과 엑스선 및 중성자 격자 간섭계로부터 얻어진 실험 데이터에 적용되었다. 그 결과 웨이블릿 전처리와 기계 학습 방법(WCNN)의 조합이 가장 효과적인 것으로 나타났다.

Ⅰ. INTRODUCTION

The conventional X-ray imaging acquires the transmission information of an object, and the contrast is generated by the X-ray attenuation of the object. Since the materials such as bone have high attenuation, they can be easily distinguished from other materials and show higher contrast in the image. Thus, it can be a valuable diagnostic tool for the dense tissues like bone, but its soft tissue contrast is still quite limited. On the other hand, the newly introduced X-ray grating interferometer provides a good opportunity to improve the soft tissue contrast not available in the conventional X-ray imaging using the phase contrast measurement capability.

In X-ray imaging, noises are generated for a variety of reasons. Such noises may deteriorate the diagnostic information and result in misdiagnosis of patients. Many of the noises are generated by the detection process, and these can be removed by simple image processing or by correcting the detector artifacts^{[1, 2]}. In the case of the differential phase X-ray image using the grating interferometers, additional artifact correction is required because the phase contrast information from the imaging is originally differential phase information, which needs line-integration process afterwards to extract the direct phase information. The measurement noises at the differential phase contrast image are accumulates in the line-integration process and cause stripe artifacts in the processed phase contrast image. Additionally, this accumulated noise may lead ring artifacts if the images are used for computed tomography reconstruction^[4-9]. Therefore, it is crucially important to develop a method to suppress the stripe artifacts in the phase contrast image resulting from the line-integration of the differential phase contrast image.

In the conventional X-ray image, digital filter methods are generally used for noise reduction^[3]. These are simple and effective for noise reduction but may induce image blurring. In the case of the stripe artifacts in the phase contrast image, which is line-integrated from the differential phase image, the noise pattern is not uniform and tend to increase to the direction of line integration. The general noise reduction method using the simple digital filteing technique is no longer effective for this kind of noise, and hence, there have been several development suggested so far. One of the proposed method is to measure the imaging in two orthogonal directions and correct the artifacts using the two sets of orthogonal data^{[4, 5]}. However, it is difficult to implement this to a real hardware system because the grating direction has to be rotated by 90 degree, and it is especially problematic for CT scan which needs multiple projections. Therefore, algorithmic methods that do not require any additional movement of the imaging hardware setup is preferred. In the algorithmic methods, the image with the stripe artifacts is multiplied by a filter in the Fourier domain and additional normalization step is followed^[6-8]. This method would be more effective when used in combination with the first method. However, in this method, we need variables to set for each image, which affects the image contrast. In order to resolve this issue, a method preserving the contrast of an image during the stripe artifact reduction was proposed^[9]. This method is fast and effective by removing the stripe artifact by combining the Wavelet and Fourier filtering. But there remains a residual noise pattern after processing the stripe artifact using the method. There is another effective phase retrieval method based on total variance(TV) regularization but it takes long time for data processing^[10].

In recent years, various methods have been introduced to remove noise by machine learning^{[11, 12]}. Noise removal process using machine learning may be effective while not significantly changing the image contrast by optimally designing the machine learning layer structure and the training dataset. Although the method using machine learning takes long time to train, it is more time-efficient to process a large amount of data than other methods because it can be used repeatedly once the training is done.

Noise reduction using machine learning has been usually applied to a fixed pattern noise. Machine learning trains data to remove the noise effectively, by using a loss function. When the noise pattern is regular or uniformly distributed, it can be more effectively removed through machine learning. On the contrary, the stripe artifacts in the phase contrast image are more randomly distributed, and the machine learning is not as simple as the fixed noise case. In this paper, we propose methods to remove the stripe artifacts using machine learning. We evaluate the suggested methods and discuss the results.

Ⅱ. MATERIAL AND METHODS

1. Dataset

There are two important things in machine learning: one is to prepare the dataset, and the other is to construct the network architecture. In order to generate a dataset with the images with stripe artifacts due to the line-integration, we divide 400 black and white images into 64×64 patches and add 16 padding to them on the left and right to obtain 46800 images with a size of 64×96. Then, these images are differentiated with respect to the X direction to obtain the differential images. Noises are added to these images, and the line-integrations are performed on the images to generate the stripe artifacts. This process is illustrated in Fig. 1. The PSNR in the differential image with the Gaussian noise was 30 dB. When the mean PSNR values of the differential images were about 30 dB, the PSNRs of the line-integrated images were very poor due to the noise accumulation. Each dataset is subject to additional preprocessing for the purpose of a simulation. The three kinds of dataset were prepared: the first one is the set of the differential phase contrast images with Gaussian noise added; the second the set of the phase contrast images with stripe artifacts due to line-integration; and the third one the set of the phase contrast images with wavelet preprocessing applied.

Fig. 1. Dataset creation process: (a) used for CNN, CCNN, PCNN. (b) used for CNNW, CNNWCNN, WCNN.

2. Network Architectures

Since the performance of machine learning depends on the network architecture, we construct the network layers as shown in Fig. 2. The layers used in this study are classified into three types: The first one adopts a single CNN; the second one uses two consecutive independent CNNs; the one adds a max pooling layer to a CNN. The third method, the pulling layer addition to a CNN, is advantageous to eliminate striped noises, ^{[13, 14]} but the image could be blurred due to the pulling. Since images become blurred as the number of layers increases, we properly chose to use 10 layers. The first CNN filter used a size of 5×5 and the rest used a size of 3×3. The machine learning training was performed 1000 times.

Fig. 2. Layer structure: (a) CNN, (b) CCNN, (c) PCNN, (d) CNNW, (e) CNNWCNN, (f) WCNN

We used Matlab (version R2017b) from Mathworks for data processing, and MatConvNet (version 1.0) for machine learning tool. The specifications of the computer is as follows: Intel Xeon CPU e5-2680 v3, 128GB RAM, and GTX 1080 SLI.

3. Performance evaluation of the pure machine learning methods

First, we evaluated the initial performances of the three pure machine learning methods in Fig. 2(a)~(c) to eliminate the stripe artifacts in the phase contrast image using machine learning. The network architecture in Fig. 2(a) (CNN) removes the noise in the differential phase contrast image by machine learning, followed by the line-integration for the phase contrast image. The one in Fig. 2(b) (CCNN) has an pose-processing step to reduce noise using machine learning. Fig. 2(c) (PCNN) has a pooling layer.

The phase contrast images, shown in Fig. 3(d)~(f), enhanced by the pure machine learning methods demonstrate no better result than the conventional methods^[4-10] to remove the stripe artifacts in the phase contrast image. Table 1 shows that the Wavelet method has the highest PSNR compared to the pure machine learning methods evaluated in this study. This is attributed to the characteristics of machine learning. In machine learning, the accuracy is improved by using the loss function and this is effective for the conventional noise or uniform noise which does not deviate greatly from the ideal value. However, the stripe artifacts in the phase contrast image are significantly different from the ideal values, and the regularity of which are not easy to find by training. Therefore, in this study, we use a complex machine learning method which combines the machine learning methods with the conventional data correction methods to further enhance the phase contrast image quality. Of the three conventional methods introduced (Phase Retrieval, ^[6-8] Two Direction, ^{[4, 5]} Wavelet^[9]), the Wavelet method are chosen for this study because the results from the phase retrieval and two direction methods largely depend on the setting parameters for the processing and are not very practical to be applied to large amounts of dataset as in our case. Moreover, they do not increase the PSNR significantly and perform worse than the Wavelet method we chose. Hence, we believe that preprocessing the dataset using he Wavelet method is the most effective, takes the least time, and robust. Although the wavelet method is effective in reducing the artifacts by optimally adjusting the decomposition level L, the wavelet type, and the damping factor, ^[9] it could introduce a new noise with a wave pattern However, this wave pattern noise has relatively uniform distribution over the image and is easier to be corrected by the machine learning methods. In this study, we try to find the complex machine learning method to combine the wavelet method and the machine learning methods for the most effective stripe artifact removal in the phase contrast image. The network architectures of the complex machine learning method tested in this study is shown in Fig. 2(d)~(f), and they are named as the CNNW, CNNWCNN, WCNN, respectively. Here, we report the simulation result by comparing them.

Fig. 3. Performance evaluation of simple machine learnings in Shepp-Logan Phantom: (a) Reference, (b) Direct Integration, (c) Wavelet (DB25, σ=6, L=6), (d) CNN, (e) CCNN, (f) PCNN.

Table 1. PSNR comparison of simple machine learning in Fig. 3.

Ⅲ. Results and Discussion

1. Simulation with Shepp-Logan Phantom

In the preliminary test in Section II, we have seen that the pure machine learning methods such as CNN, CCNN, and PCNN are no better than the wavelet method. Therefore, we have configured other combination of the methods for further reducing the artifacts. We use the wavet method as a reference and have configured and compared the following noise reduction schemes: Wavelet, Wavelet + Phase Retrieval (W + PR), Wavelet + Two Direction (W + TD), CNNW, CNNWCNN, WCNN. We have used the Shepp-Logan Phantom image shown in Fig. 4(a) as a simple case for the test. The original Shepp-Logan Phantom image is differentiated to the x-direction to emulate the differential phase contrast image to be used as the input for the schemes or line-integration. The stripe artifacts in the Shepp-Logan phantom image become dominant to the right direction of the phantom as shown in Fig. 4(b), which is due to the noise accumulation to the integration direction. The effectiveness of the noise removal for each noise reduction scheme and it turns out that the WCNN in Fig. 4(h) is the best from the qualitative visual evaluation. The quantitative evaluation is summarized in Table 2 with the PSNR values for each scheme. The numerical simulation shows that WCNN is the best model for the stripe artifact reduction. The profiles from the WCNN and the Wavelet have been plotted in Fig. 6(a) and they have good agreement with the origitnal profile but the WCNN is slightly better to the boundaries. We conclude that the WCNN is the most effective among the schemes compared in this study.

Fig. 4. Comparison image of methods using wavelet in Shepp-Logan Phantom(512X512): (a) Reference, (b) Direct Integration, (c) W+PR, (d) W+TD, (e) CNNW, (f) CNNWCNN, (g) Wavelet (DB25, σ=6, L=6), (h) WCNN.

Table 2. PSNR comparison of simulation in Fig. 4 and Fig. 5

Fig. 6. Profile of simulation : (a) Shepp-Logan Phantom, (b) Lena.

2. Simulation with Lena

We have used the Lena image as shown in Fig. 5(a) to test the schemes for more complex case than the previous Shepp-Logan Phantom. The stripe artifact of the Lena image is severer than the simple Shepp-Logan Phantom after line-integrating the differential Lena image with noise added as shown in Fig. 5(b). Also, the contrast after the noise reduction using the existing methods such as Phase Retrieval (PR) and Two Direction(TD) is deteriorated as shown in Fig. 5(c) and 5(d). The noise reduction effectiveness in the Lena image is more difficult to evaluate than Shepp-Logan case because of its more complicated features but it is more realistic because many experimental data may include such complicated features. In Fig. 5, we can see that the wavelet pattern noise is larger in the Lena images than the Shepp- Logan Phantom image. WCNN is the most effective as we compared the PSNR in Table 2. Fig. 6(b) shows the profile comparison of the schemes in the Lena image and WCNN follows the reference profile better than the others.

Fig. 5. Comparison image of methods using wavelet in Lena(512X512): (a) Reference, (b) Direct Integration, (c) W+PR, (d) W+TD, (e) CNNW, (f) CNNWCNN, (g) Wavelet (DB25, σ=6, L=6), (h) WCNN.

However, we notice that the wavelet method is still effective when the noise level is significantly lower because the wave pattern noise becomes insignificant for the lower noise level. We believe that machine learning provides a way to reduce the regular patterns by training and further reduce the artifacts in combination with the wavelet. Therefore, we conclude that it contributes to the improvement of image quality as well as to the noise reduction.

3. Applications

In the case of actual experiment image data, it is difficult to apply the conventional machine learning methods because the degree of noise is generally unknown unlike the previous simulation cases. Hence, we focused on removing the wavelet pattern noise after reducing the stripe artifacts with the wavelet method as in the proposed model. We evaluate the validity of this approach in this simulation.

The three samples were used for the experimental data and they are the cylinders, plant stems, and a coin. The cylinders and plant stems were measured using the X-ray grating interferometer^[16], and the coin image was acquired using neutrons^[17]. Since the neutron image have higher noises in our experiment, the data was considered as a high noise case to evaluate the effect of noise level.

Fig. 7 shows the test results for the x-ray phase contrast image of the cylinder sample. In Fig. 7(b), (c), the stripe artifacts are not removed by the CNN and CCNN. In Fig. 7(e), (f), the CNNW and CNNWCNN performed relatively well with the numerical simulations, but were not as effective with this real data. The result by WCNN shown in Fig. 7(g) shows no image distortion and a significant stripe artifact reduction. WCNN improve the overall image quality when compared with the simple wavelet method in Fig. 7(d). Thus, we conclude the WCNN method is as effective to remove the artifacts for the real data case as for the numerical case.

Fig. 7. Comparison between Wavelet and machine learning methods in actual X-ray experiment of cylinder sample. (a) Direct Integration, (b) SCNN, (c) CCNN, (d) Wavelet (DB25, σ=6, L=6), (e) CNNW, (f) CNNWCNN, (g) WCNN.

In Fig. 8, we have compared the simple Wavelet, and WCNN for the X-ray phase contrast image of a plant stem and the neutron phase contrast image of a coin. The enlarged view in Fig. 8(b) and 8(c) shows that the WCNN removes the wavelet pattern effectively. Fig. 8(d)~(f) shows effectiveness of the WCNN for the neutron phase contrast image of a coin sample. The stripe artifact has been reduced and the image sharpness and contrast has been improved. Therefore, we conclude that the WCNN method is effective for the high noise level as in this case.

Fig. 8. Comparison between (b), (e) Wavelet (DB25, σ=6, L=6) and (c), (f) WCNN in actual experiment image. (a), (b), (c) is X-ray image of plant stem and (d), (e), (f) is neutron image of coin.

Ⅳ. CONCLUSION

In this study, we have seen the effectiveness of the six representative machine learning structures for reduction of the stripe artifacts in the phase contrast image generated by the one-directional line-integration of the differential phase contrast image as shown in Fig. 2.

The CNN can be applied to a differential phase contrast image or its phase contrast image after the line integration. The first structure (SCNN) in Fig. 2 have the noise reduction process using the machine learning for the differential phase image; the second structure (CCNN) both for the differential phase contrast image and the phase contrast image; the third structure (PCNN) have a pooling layer in the CNN for the differential phase contrast image. In addition to the above three structures, we have designed the other three noise reduction schemes in combination of the wavelet Fourier filtering and the machine learning. The CNNW has the wavelet filtering after the CNN for the phase contrast image; the CNNWCNN has additional CNN after the wavelet filtering to the CNNW; the WCNN has the wavelet filtering before the CNN. The wavelet filtering applied to the differential phase contrast image is not included here because the filtering effect in the step does not show any improvement.

First, we have evaluated the possibility of CNN, CCNN, PCNN and found that these machine learning methods are less effective for artifact removal than the simple wavelet. Since there is a disadvantage that wavelet pattern noise is generated when the wavelet method applied to high level noise, we have combined the wavelet and machine learning methods to remove the stripe artifacts. The wavelet pattern noise, which is a uniformly distributed noise, could be effectively removed using machine learning. We have compared the CNNW, CNNWCNN, WCNN and conventional methods, which are Two Direction, Phase Retrieval and Wavelet Fourier filtering, on the Shepp-Logan phantom and Lena image. In all cases, the WCNN method preprocessing the dataset using the wavelet had a PSNR value about 10% higher than that of the conventional Wavelet method. Furthermore, we have evaluated these methods to the actual experimental images, which are the X-ray phase contrast images (cylinders, stem) and neutron phase images (coin), and found out that the WCNN model is the most suitable.

At present, it is effective to use hybrid method, which is a combination of the wavelet and machine learning. It is expected that it is possible to further reduce the stripe artifacts using more sophisticated methods such as deeper machine learning layer structures or by improving the dataset. We have seen that the wavelet operation can be improved by machine learning. Furthermore, this research can be extended to the phase contrast tomography where the stripe artifact reduction may be more important because the artifacts can become severe during the reconstruction process.