1. Introduction
With the increasing demand for product safety and quality, internal crack inspection of products is getting more and more critical since cracks are invisible and they can lead to fatal crashes or damage. With the ability to penetrate materials and be attenuated according to the material’s thickness and density, X-ray inspection systems [1-3] are widely used for the identification and evaluation of internal cracks in many industrial products nowadays. In a conventional inspection system, the X-ray images are manually examined by radiologists in order to check if there are cracks in the product. However, the radiologists can get tired after long periods of examination. Furthermore, the inspection results can vary with respect to the skill and alertness of the operator. Therefore, automatic inspection systems [3-5] based on image processing have been developed to improve inspection results and reduce the workload of operators.
In order to detect cracks in X-ray images automatically, Shao et al. [3] proposed a segmentation method based on double-thresholding and fuzzy set theory to automatically detect cracks from the X-ray images. The system includes three main components: preprocessing, thresholding segmentation and post-processing. By applying this method, small cracks can be detected. However, it has limitations in detecting cracks with non-uniform back-grounds. Moreover, it fails to detect various sizes of crack. H. Y. Chai, et al. [6] proposed a method using GLCM (Gray-Level Co-occurrence Matrix) [7] computerized techniques to detect femur bone fracture in X-ray images automatically. Since this method uses GLCM, it is sensitive to image noise. In [8], a novel LBP (Local Binary Pattern) [9] based operator for pavement crack detection was proposed by Y. Hu, et al. In this approach, local neighbors were firstly classified into smooth area and rough area (crack detection is only performed on the rough area). Then, local patterns extracted by LBPriP,R [10] were redefined into five subclasses and pixels with patterns that belong to the edge or corner subclasses were considered as crack candidates. This approach is able to detect those small sizes of cracks fast. However, this method is sensitive to the image edge resulting in a high percentage of false alarm. Recently, Peng et al. [5] proposed a crack filter based on a Gaussian model and multiscale analysis to inspect cracks in metal blades. To enhance the crack regions in the X-ray image, a function based on a Gaussian model was defined to represent the cross section of the cracks. The filter was applied in the x-direction and y-direction in the image to detect different directions of crack. A Gaussian function with different scales was employed to analyze different sizes of cracks. The algorithm showed good performance in detecting cracks of various sizes in X-ray images with noise and non-uniform intensity problems. However, because various scales of Gaussian functions were adopted, the processing was time-consuming.
The motivation of this paper is to develop a robust crack filter that can quickly detect cracks of various sizes in X-ray images with noise and non-uniform intensity problems. By analyzing the characteristics of the cracks, the new crack filter takes advantage of the image’s local gray level and its variation to detect cracks by enhancing the crack regions and attenuating the flat regions in the image. First, the local gray level variation is estimated by calculating the Maximum Neighboring Gray Level Variation (MNGLV); this can overcome the problems of noise, non-uniform intensity and edge response in the X-ray image. Second, in order to detect cracks of various sizes, this paper proposes using different neighboring distances to construct an image pyramid for multiscale analysis. By selecting the maximum response from the image pyramid, the filter is able to detect cracks of various sizes. Owning to the utilization of the local gray level variation and multiscale analysis, the proposed crack filter is able to detect cracks of various sizes and it is robust to image noise and non-uniform intensity problems.
The main contributions of this paper are as follows:
• A new method called MNGLV is proposed to estimate the local gray level variation. It is robust to image noise and non-uniform intensity problems. Moreover, this method avoids responding to image edges.
• This paper proposes using different neighboring distances to construct an image pyramid for multiscale analysis, which enables the proposed filter to detect cracks of various sizes.
The remainder of this paper is organized as follows: Section 2 gives a brief introduction of the 2D X-ray system and Section 3 presents the proposed crack filter. The experimental results are shown in Section 4 and the conclusion is presented in Section 5.
2. A typical 2D X-ray System
A typical 2D X-ray inspection system is shown in Fig. 1. First, a cone of X-rays is generated by the source (tube) and detected by the image intensifier, which converts the X-rays into a visible radioscopic image. The gray image is then captured by the camera behind the image intensifier and sent to the computer for further analysis. Inspection is conducted by moving the object into the X-ray cone.
Fig. 1.A typical 2D X-ray system
Due to the properties of X-rays, the sample inside the X-ray cone absorbs the X-ray radiation according to the density, atomic structure and thickness of the sample. The thicker or / and denser the sample is, the more X-ray radiation is absorbed. As a result, a thicker or/and denser sample will generate a darker area in the image corresponding to the higher X-ray absorption. Therefore, defects are characterized in the X-ray images by local changes in the image intensity, resulting in the corresponding local discontinuities in the gray values of the acquired image [11]. When a defect occurs in a product (such as a crack in a casting), the resulting X-ray image will show some areas with local gray level changes in the image, which tells us that something has happened inside the product. In most of the cases, the sample for inspection usually has a different thickness or density inside. Due to the attenuation properties of the X-ray, the images captured by the system usually show non-uniform intensity, which is a challenge for automatic defect detection. Fig. 2 shows an X-ray image that was acquired from part of a metal ring sample. As we can see, a crack region shows a discontinuous gray value distribution in the image. Meanwhile, the intensity in the image is non-uniform.
Fig. 2.A metal ring X-ray image
3. The Proposed Filter
As presented in Section 2, the high gray level and high local gray level variation are two characteristics of a crack region in an X-ray image. Based on these characteristics, a new crack filter is proposed in this paper to detect crack regions in the image automatically. First, the local gray level variation is estimated by a new method. The output of the proposed crack filter is then defined by combining the gray level and its local variation. Finally, an image pyramid is constructed by using different neighboring distances for multiscale analysis.
3.1 Estimation of local gray level variation
One of the most popular approaches to estimate the gray level variation is using the Hessian matrix, which is defined as:
where Lxx and Lyy denote the second partial derivatives of the image in the x and y directions. Lxy is the mixed partial second derivative of the image in the x and y directions. Lxx, Lyy and Lxy are defined as:
where I(i, j) is the gray level of the image in the position of (i, j).
As shown in (2) ~ (4), Lxx and Lyy are the gray level variations between the pixel (i, j) and its four neighbors. Lxy denotes the gray level variation of the neighboring pixels. For multiscale analysis, the image is down sampled and the Gaussian function with various scales σi is applied to generate an image pyramid. Lxx, Lyy and Lxy are then calculated in the pyramid for multiscale analysis. A typical algorithm based on this gray level variation is the Laplacian of Gaussian (LoG) [12], which defines the output of the operator as the summation of Lxx and Lyy in the image pyramid.
The Hessian matrix has shown good performance in gray level variation estimation. However, since this method calculates the gray level variation between a point and its neighbors, it also gives a response to image edges. In multiscale analysis, when the Gaussian function is applied to construct an image pyramid, it is time-consuming.
To avoid the response of the image edge and speed up processing, a new method called Maximum Neighboring Gray Level Variation (MNGLV) is proposed to estimate the local gray level variation. First, an 8-neighboring model for a point is constructed by defining a neighboring distance di (Fig. 3).
Fig. 3.The 8-neighboring model for gray level variation estimation
Based on the 8-neighboring model, the MNGLV is defined as follows:
where j = 1, 2, 3, and 4. g(nj) is the gray level of the neighboring point nj and g(c0)is the gray level of the center point. As we can see from (5), gray level variation is obtained from four directions (horizontal, vertical, and two diagonal directions). For each MNGLVj, the gray level variation is estimated in one direction by calculating the gray level difference between C0 and the neighboring point (nj), which has a higher gray level. Since only the higher gray level neighboring point in one direction is used, the MNGLV can avoid responding to the image edge.
3.2 Output of the proposed filter
As presented in sub-section 3.1, the MNGLVj represents the local gray level variation in one direction. Since the crack region in the X-ray image has the characteristics of a high gray level and high local gray level variation, the output of the proposed crack filter is defined as the combination of the gray level and the summation of MNGLVj:
where A is a user defined constant used to visualize the filtered result and k∈(0,1) is a user defined constant used to control the response of the gray level variation. For a small value of k, the output of the filter is sensitive to the local gray level variation. Consequently, the filter can give a response to low contrast crack regions. However, it is also sensitive to image noise. Therefore, k is a tradeoff between crack response and noise sensitivity. By properly choosing the value of k, this method can respond to the low contrast crack regions while constraining the response to image noise. Since k can be modified according to different applications, the utilization of k makes the proposed method flexible. Also, as shown in Fig.3, due to the fact that four directions (horizontal, vertical, and two diagonal directions) are utilized, this method can analyze the local gray level variation precisely. Therefore, the usage of four directions enables the proposed method to generate good crack detection result. As we can see from (6), only the high gray level of the center point and high value of MNGLV can generate high output. Therefore, the filter can obtain crack regions and remove flat regions in the image. Note that the gray level of the image is normalized between 0~1 before applying the filter. Furthermore, for the detection of dark cracks, a gray level inversion operation should be implemented before applying the filter.
3.3 Multiscale analysis
As shown in Fig. 4, cracks in the X-ray image have various sizes. To detect different sizes of crack, the crack filter should be invariant to crack size.
Fig. 4.A screw gasket with different sizes of crack
In order to detect different sizes of crack, Peng et al. [5] proposed a Gaussian model based crack filter which uses different scales to construct an image pyramid for multiscale analysis. However, the Gaussian convolution is time-consuming. In order to detect different sizes of crack and speed up the processing, this paper proposes to use different neighboring distances di to construct an image pyramid for multiscale analysis. The procedure for constructing the image pyramid is shown in Fig. 5.
Fig. 5.Multiscale analysis of the proposed filter
Suppose that the minimum and the maximum sizes of the cracks are Smin and Smax, respectively, the neighboring distance di in the image pyramid is determined by:
where Sp is calculated by:
where N (N≤ Smax−Smin) is the number of neighboring distances for multiscale analysis.
As shown in Fig. 5, the number of neighboring distances is first defined and the 8-neighboring models are determined by using Eq. (7) and (8). For a defined neighboring distance di, the filter is applied to the input image based on the 8-neighboring model, which results in an output image Oi. Obviously, a small distance (di) can be used to detect small size of cracks whereas it is not able to detect big size of cracks because it generates a small value of MNGLVj (see Eq. 5). For a large distance (di), it can estimate the MNGLVj properly for the big size cracks whereas it may fail to calculate the MNGLVj correctly for those small sizes of cracks in a complex background image. Consequently, the neighboring distance di who matches the size of the crack will produce the maximum response. Therefore, the final output of the crack filter is generated by selecting the maximal response points of the output images (O1, O2 …On). Since the proposed method uses different distance (di) to analyze different size of cracks, it is more effective than the method in [5], which needs to convolute different scales of Gaussian coefficients with the input image for multiscale analysis.
For a large value of N, more neighboring distances will be used and thus the analysis is more precise. However, more processing time is needed. On the other hand, a small value of N results in less neighboring distances for the multiscale analysis, and therefore the processing time is reduced. However, this degrades the analysis precision. Therefore, N is a tradeoff between the processing time and the analysis precision. It can be determined with respect to the implementation.
Fig. 6 is the filtering result of Fig. 4 by using the proposed method. As we can see, the proposed crack filter can detect different sizes of cracks in the image with non-uniform intensity and noise.
Fig. 6.The filtering result of Fig. 4
4. Experimental Results
4.1 Image dataset
To test the performance of the proposed crack filter, a set of X-ray images captured from metal rings were collected using an X-ray machine (TVX-IMT160) provided by an X-ray company (Techvalley Co., Ltd.). 200 images (including 111 cracks) were captured by the machine. The images were 1000-by-1000 pixels and stored in an 8-bit JPG format. The images include noise and non-uniform intensity problem. Fig. 7 shows some image samples.
Fig. 7.Image samples of the dataset
4.2 The evaluation methodology
The performance of the proposed crack filter was tested by applying the filter to an automatic crack inspection system, which is similar to the system presented in [5]. The system includes three main steps (Fig. 8): 1) preprocessing; 2) crack detection; and 3) post-processing.
Fig. 8.The automatic crack inspection system
As shown in Fig. 8, in the pre-processing step, the low pass filter (median value filter, with a window size of 3-by-3) is applied to partially reduce the image noise. The OTSU method [13] is used to get a binary image after the low pass filter processing. The region of interest (ROI) is then obtained by region growing. Finally, the contrast of the ROI is enhanced by the histogram stretch method [5]. In the second step, the proposed crack filter is applied to the pre-processed image to get a crack enhanced image (Fig. 6). In the post-processing step, a threshold (100 for the proposed filter and the Gaussian model based filter) is first used to segment the candidate crack regions from the crack enhanced image. The features of the candidate crack regions (as described in [5]) are then extracted and used to reject the fake crack regions. In the system, the candidate cracks whose sizes are greater than 2000 pixels or less than 70 pixels are regarded as fake cracks.
The performance of the proposed crack filter is evaluated by comparing the detection accuracy, false detection ratio, and processing speed with those of the Gaussian model based crack filter [5] and the LBP model based method [8]. The detection accuracy and the false detection ratio are defined similarly to [5]:
where TP (true positive) denotes the number of cracks that are correctly detected, FP (false positive) is the number of normal regions that are incorrectly detected as crack regions, and CN is number of true cracks. Note that TP, FP and CN were inspected by two radiologists from the X-ray machine company and are regarded as an accurate representation. As described in Eq. (9) and (10), a good system can achieve a high detection accuracy and low false detection ratio. The proposed crack filter and the filters presented in [5] and [8] are applied to “Step 2” of the system (Fig.8) to obtain the detection accuracy and false detection ratio, respectively. Note that for the method in [8], the thresholding processing in the post-processing step is skipped since the pattern selection in [8] directly generates a binary image.
For evaluating the processing speed, we defined the speeding-up as follows:
where TRef is the processing time of the Gaussian model based crack filter or the LBP model based method, and TMNGLV is the processing time of the proposed crack filter. Therefore, the greater the SP is, the faster the relative speed of the proposed filter is. Table 1 shows the experimental environment for the following test.
Table 1.The experimental environment
4.3 Inspection results
Table 2 shows the parameters for the proposed crack filter, which provide the best inspection result for the dataset.
Table 2:The parameters for the proposed filter
The inspection results of the proposed crack filter, the Gaussian model based filter [5], and the LBP model based method [8] are shown in Table 3. As we can see, the proposed method outperforms the Gaussian model based filter and the LBP model based method with respect to the detection accuracy and false detection ratio. Since the proposed method estimates the local gray level variation in four directions (horizontal, vertical, and two diagonal directions, as shown in Fig.3), it is more robust in detecting different directions of cracks than the Gaussian model based method [5], which performs the filter in two directions (horizontal and vertical directions). And due to the combination of the local gray level variation and the intensity information, the proposed crack filter is more sensitive to the corresponding crack regions. Therefore, the proposed method achieved 7.21% higher detection accuracy than the Gaussian model based filter and 29.73% higher detection accuracy than the LBP model based method. Furthermore, by adding k to control the response of the gray level variation, the proposed method is more flexible than the other two methods. Hence, the proposed method produces less fake crack regions than the other two methods do. As we can see from Table 3, the proposed method outperformed the Gaussian model based crack filter and LBP model based method by 4.49% and 25.22% with respect to the false detection ratio. For the LBP model based method, because it detects the crack regions by only using the LBP patterns, it fails to estimate the local gray level variations precisely. Moreover, since the LBP model based method does not utilize a multiscale analysis, it is not robust to the sizes of crack regions. Because of these reasons, the LBP model based method achieves the lowest detection accuracy comparing to the other two methods. And since the LBP patterns are sensitive to the image edge, the method produces a number of fake crack regions, which leads to a higher false ratio than the other two methods do.
Table 3.The inspection results of the proposed crack filter and Gaussian model based filter
The processing time of the methods is shown in Table 4. Note that the time shown in Table 4 only denotes the processing time of the methods. For a fair comparison, all the unnecessary processes of the operating system were ended during testing. As we can see from Table 4, the proposed method was 2.63 times faster than the Gaussian model based crack filter and 1.26 times faster than the LBP model based method. Instead of using the Gaussian model, the proposed method used different neighboring distances di for multiscale analysis. Since the Gaussian convolution is avoided, the processing time of the proposed method is speeded up. For the LBP model based method, since it does not use a multiscale analysis, it is also faster than the Gaussian model based filter. However, since the rotation invariant LBP pattern (LBPriP,R) is used, the method needs to calculate new positions for the circular neighbors for each pixel, and needs to compute the new gray values by interpolation. As a result, this method consumes similar time with the proposed filter even it does not use multiscale analysis.
Table 4.The processing time of proposed filter and the Gaussian model based filter
5. Conclusion
A robust crack filter based on local gray level variation and multiscale analysis was presented in this paper. The proposed crack filter makes use of the image gray level and its local variation information to detect the cracks in the X-ray image. By use of local gray level variation, the proposed filter is robust to image noises and non-uniform intensity problems in X-ray images. By defining different neighboring distances for multiscale analysis, the proposed filter is able to detect cracks of various sizes rapidly
참고문헌
- J. Xu, T. Liu, X. M. Yin, B. S. Wong, Syahrom B. Hassan, Automatic X-ray Crack Inspection for Aircraft Wing Fastener Holes, 2nd International Symposium on NDT in Aerospace 2010, pp. 1-8.
- J. Schors, K. W. Harbich, M. P. Hentschel, A. Lange, Non-Destructive Micro Crack Detection in Modern Materials, 9th ECNDT, September 2006.
- Z. H. Shao, L. L. Liang, Automatic Segmentation of Cracks in X-ray Image Based on OTSU and Fuzzy Sets, 3rd International Congress on Image and Signal Processing, 2010, pp. 1824-1827
- W. Chen, Z. Miao and D. Ming, Automated Inspection using X-ray Imaging, IEEE International Conference on Trust, Security and Privacy in Computing and Communications, Nov.2011, pp. 1769-1772
- S. H. Peng, H. S. Kim, H. D. Nam, Metal Crack Detection in X-ray Images Based on Local Brightness Variation and Multi-scale Analysis, Information Journal, Vol. 15, No. 5, May 2012, pp. 1961-1972
- H. Y. Chai, L. K. Wee, T. T. Swee, S. H. Salleh, A. K. Ariff and Kamarulafizam, Gray-Level Co-occurrence Matrix Bone Fracture Detection, American Journal of Applied Sciences, 2011, pp. 26-32
- R. M. Haralick, Statistical and structural approaches to texture, Proceedings of the IEEE, 1979, pp. 786-804
- Y. Hu, C. X. Zhao, A Local Binary Pattern Based Methods for Pavement Crack Detection, Journal of Pattern Recognition Research, 2010, pp. 140-147
- T. Ojala, M. Pietikainen, and D. Harwood, A Comparative Study of Texture Measures with Classification Based on Feature Distributions, Pattern recognition, 29(1996), pp. 51-59 https://doi.org/10.1016/0031-3203(95)00067-4
- T. Ojala, M. Pietikainen, and T. Maenpaa, Multiresolution, Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns, IEEE Transactions on Pattern Analysis and MachineIntelligence, 24(2002), pages 971–987 https://doi.org/10.1109/TPAMI.2002.1017623
- X. Li, S. K. Tso, X. Guan, and Q. Huang, “Improving Automatic Detection of Defects in Castings by Applying Wavelet Technique”, IEEE Transactions on Industrial Electronics, Vol. 53, No.6, Dec. 2006, pp.1927-1934 https://doi.org/10.1109/TIE.2006.885448
- T. Lindeberg, “Feature detection with automatic scale selection”, International Journal on Computer Vision, 1998, pp. 77-116
- N. Otsu, A Threshold Selection Method from Gray-Level Histogram, IEEE Transactions on System Man Cybernetics, volume SMC-9, No.1, 1979, pages 62-66