3D Inspection by Registration of CT and Dual X-ray Images

Abstract

Computed tomography (CT) can completely digitize the interior and the exterior of nearly any object without any destruction. Generally, the resolution for industrial CT is below a few microns. The industrial CT scanning, however, has a limitation because it requires long measuring and processing time. Whereas, 2D X-ray imaging is fast. In this paper, we propose a novel concept of 3D non-destructive inspection technique using the advantages of both micro-CT and dual X-ray images. After registering the master object’s CT data and the sample objects’ dual X-ray images, 3D non-destructive inspection is possible by analyzing the matching results. Calculation for the registration is accelerated by parallel computing using graphics processing unit (GPU).

Introduction

Recently, computed tomography (CT) has been used for non-destructive inspection. The rapid development of CT scanning hardware and software, CT became one of the major inspection solutions for non-destructive inspection. Because non-destructive inspection does not damage the product, it is beneficial in time and money in product quality testing. Most of all, this non-destructive inspection is important if the product should be preserved the original condition. CT scanning can detect porosities, voids, flaws, inclusions, and inside castings in three-dimension. However, because 3D CT scanning takes much time, it can be used for only the inspection of samples. To overcome this problem, we propose a new method to inspect all the products by using dual X-ray imaging. The dual X-ray or stereo X-ray imaging uses two X-ray images from the orthogonal views. Accurate calibration of 2D-3D registration is also required. Once the 3D CT scan of a master is registered to the dual X-ray images of a sample, the matching score is computed to inspect the sample.

 

Material and Methods

The entire process of the proposed non-destructive inspection using 2D-3D registration is shown in Fig. 1. The data necessary for the 3D non-destructive inspection using registration of micro-scale radiographic images include (1) micro-CT scan data of the master object and (2) two orthogonal X-ray images of a sample object. First, the master object’s 3D CT scan data are prepared. The master object’s sectional images are obtained from a micro-CT scanner. Next, dual (or stereo) X-ray images of a sample object are taken from two orthogonal positions. In our hardware setting, we captured the dual X-ray images by rotating the sample object by 90 degrees. With these two datasets, 3D position of the sample object is calculated by registering the CT data of the master object to the dual X-ray images of the sample object. After the registration, the sample object is inspected by checking if it produces the same images with the master object or not. The conventions are as follows; I1: X-ray image at the first position and I2: X-ray image at the second position of dual X-ray imaging, V1 and V2: digitally reconstructed radiographs with dual X-ray imaging at each position. The details of registration and inspection with micro-scale X-rays are described in the following sections.

Fig. 1.Workflow of non-destructive inspection using 2D-3D registration of micro-scale radiographic images.

Calibration of Dual X-ray Imaging System

The 3D geometrical configurations of dual X-ray imaging system are obtained by calibration process (1, 2). Calibration of X-ray imaging geometry is aimed at determining the relationship between the image coordinate system and the world coordinate system of X-ray systems. Considering the parameters necessary when creating virtual radiographs, following parameters are determined: (1) magnifications between radiographs and detector, (2) relative position between detector and X-ray source, and (3) rotation axis of dual X-ray system. A calibration cube and a calibration frame are designed for the calibration processes (Fig. 2). All the X-ray images in the study were preprocessed with distortion compensation. First, the magnifications between pixel coordinates and world coordinates are computed from the known distance lattice points of calibration panel and their corresponding points in the image. Next, relative position between the X-ray source and detector is estimated with the calibration points on the calibration cube’s two parallel planes by solving a least square problem (Fig. 2(A)). Lastly, rotation axis for dual X-ray system is estimated with the calibration frame (Fig. 2(B)). Because we already know about the dimensions of the calibration points of the calibration cube and the calibration frame, we can determine the parameters of the actual dual X-ray imaging system. During the calibration, the optimal parameters of the actual dual X-ray imaging system are computed by minimizing the discrepancy between the estimated calibration points and their corresponding actual points in the X-ray images in an iterative manner. Each calibration process for this study is described in detail in (1).

Fig. 2.(A) Calibration cube and (B) calibration frame designed for calibration of (C) dual X-ray imaging system, and (D) dual X-ray images of calibration frame.

DRR generation and its acceleration

Digitally Reconstructed Radiography (DRR) is a process of generating a virtual X-ray image using a virtual X-ray imaging system (3). The virtually generated dual X-ray images or DRRs are rendered with a pair of X-ray sources and detectors. To calculate the DRRs of the master object, projections are performed using virtual X-ray sources and virtual detectors. The information of the X-ray source S1 (or S2) and the detector D1 (or D2) reproduced through the calibration in the previous section is used. For the vertices of the triangular facets of individual models, projection lines are calculated from SAP (SLat) and the points at which the projection lines intersect with the DAP (DLat) plane are determined. Generally, volume rendering (4) by ray casting is time-consuming process. Each pixel of DRR is calculated by accumulating intensities of CT data through which each ray passes from the X-ray source to the corresponding detector. DRR generation is regarded as the bottle-neck process in 2D-3D registration (5) because it repeatedly requires a volume rendering at each iteration of the registration. Thus, creation time of DRR is directly related to the speed of the registration. In order to accelerate DRR calculation, we used a parallel computing by using GPU instead of CPU. GPU is known as much faster in parallel computing by parallelizing it with many simple and repeatable calculations. We implemented GPU computing of DRR by using VTKEdge (Kitware Inc., USA) (6). VTKEdge is a supplementary library of VTK (7), which is a widely used open-source software system for 3D computer graphics, image processing and visualization. VTKEdge uses a graphic card of GeForce 8 series and above for parallel computing (NVIDIA Co., USA) (8).

Similarity measurement

Similarity measurement is a process to make the degree of similarity, between DRR images V1 (or V2) and actual X-ray images I1 (or I2), into values. Similarity metrics are criteria to measure similarities between two images (1). The value of a similarity metric becomes optimal when the pieces of pixel information on the two images being compared are maximally similar to each other. Such similarity metrics include mean squares metric, normalized correlation coefficient, mutual information, correlation coefficient histogram, gradient difference metric, and etc. Among the various metrics, we selected the normalized correlation coefficient (NCC), which shows the steepest changes according to the alignment of two input images.

Where Ai : ith pixel of image A, Bi : i th pixel of image B, and N: number of pixels considered

Model transformation

The parameters used to define the poses of the models are three translation values and three rotation angles. Although translations can be made by simply increasing or decreasing the values of individual variables, there are several points to be considered in relation to rotations. Since having even one less parameter confers advantageous when optimizing for registration, a rotation method using Euler angles was selected from the many possible methods of 3D rotation. Methods to express 3D rotations include rotation matrixes, rotations around arbitrary axis, and quaternions etc. Rotations made using Euler angles perform rotations in sequence around x axis, y axis and z axis by the rotation angles of θ1, θ2 and θ3 respectively to express 3D rotations and many conventions exist relating to the order of the central axes. The x-y-z convention was selected among these, which means that rotations are performed in the order of x axis, y axis and z axis by the rotation angles of θ1, θ2 and θ3 respectively. Rigid transform using Euler angle is expressed as Eq. (2).

where,

(x, y, z): original coordinate vectors,

(x’, y’, z’): transformed coordinate vectors,

T: translation vector,

R: rotation matrix defined by

R=Rz Ry Rx (x-y-z convention),

(x’, y’, z’): transformed coordinate vectors,

T: translation vector,

R: rotation matrix defined by

R = Rz Ry Rx (x-y-z convention),

θ1 (or θ2, θ3): rotation angle around x-axis(or y-axis, z-axis)

In cases where rotations are accumulated, problems will occur in performing new rotations or in calculating rotation angles from the initial position if rotations using Euler angles only are performed. Rotation matrixes were used to solve these problems. Conversions to continuously multiply the current matrix by the rotation matrix obtained from Euler angles every time a rotation is made and then calculate the Euler angle from the final rotation matrix were performed. The conversion formula between the rotation matrix R and the Euler angles Θ1, θ2, θ3 is as follows:

- Euler angle to Rotation matrix:

- Rotation matrix to Euler angle:

When performing registrations, providing an appropriate initial pose helps stable and fast calculations. To facilitate the designation of desired initial values for 3D rotations and translations of each model, an intuitive graphical user interface was implemented.

6 kinematic parameters optimization

The final stage of the 2D-3D registration is the optimization of the six kinematic parameters (three rotations and three translations). That is, the goal is to obtain the six kinematic parameters that optimize the similarity measure values between the master object’s virtually projected images V1 (or V2) and the sample object’s actual radiographs I1 (or I2). In each iteration process, the similarity metrics, between actual radiographs I1 (or I2) and DRRs created after transformations by hypothesized poses V1 (or V2), are calculated. Then, optimizations are performed to minimize the objective function defined by the similarity metrics, as shown below. Finally, parameters t* and r* for a pose are calculated through repeated iterations.

(t* , r* )=arg min E(t, r)

where (NCC)1: similarity metric for V1 and I1,

(NCC)2: similarity metric for V2 and I2,

t: translation vector, r: Euler angles.

Among various optimization methods, simulated annealing was chosen to use in this study (1, 9). Simulated annealing is a probabilistic and heuristic approach that introduces the concept of annealing, i.e. the heating of metal materials followed by slow cooling, into optimization problems. Simulated annealing (SA) uses virtual temperatures in order to avoid local minima. When searching for the minimum value of objective functions, SA searches not only downward but also upward, with the probability of logarithmically distributed random variables being proportional to temperature. As iterations are repeated, virtual temperatures are slowly reduced and accordingly, the possibility of going upstream above the minimization is also reduced. SA finishes searches when the difference between the best and the worst error scores in each period has become smaller than the specified threshold. In the implemented SA, the virtual temperature was decreased by multiplying the temperature at a particular stage by 0.9 per 10 iterations.

Non-destructive inspection with 2D-3D registration

In the previous sections, we described how the master model’s CT data are aligned to the sample model’s dual X-ray images. Here, it is assumed that the master model and the sample model are of the same shape. Let the master model be M, and the sample model be S. After the registration process, the optimal rotation and translation parameters for M are determined. The M’s optimal 3D position indicates that M is positioned at the same position of S when S’s dual X-ray images are captured. At this position, the DRR of M and the actual X-ray images of S should be exactly same. Although there exists a little bit of difference between DRR and actual X-ray image even if they were from the same model, the difference between DRR and actual X-ray image can be used for inspection. Thus, the average similarity metric value, E(t* , r* ) in Eq. (5), can be used as a metric which determines whether the internal/external shape of S is same with M or not. If the similarity metric is out of the range of normal models, S can be considered to be defective. The threshold value, , to decide normality or defectiveness is empirically determined by experiments. In the experiments for , we check the metric value with the CT data of M and the dual X-ray images of the same model M in many times. In this sense, the similarity metric value, E(t* , r* ) in Eq. (5), is a kind of matching score in inspecting the defectiveness of sample models. Once we prepared CT scan data of M, S’s internal/external shape or state can be inspected with only capturing dual X-ray images. If the matching score is below , S is regarded to be internally/externally different from M, and S is determined to be defective. Moreover, the defective part can be marked in the X-ray images by detecting the different region in the comparing images by pixel-to-pixel checking algorithm.

 

Results and Discussion

Fig. 3 shows the processes of 2D-3D registration step by step. In the top row, an X-ray image from dual X-ray images of a sample model is processed. From the original image, distortion is compensated, and then prepared to be compared with the DRR of master model. In the second row in Fig. 3, the optimal 3D pose of master model’s CT data is determined by registration of master models’ DRR and the sample model’s pre-processed image. Fig. 4 shows the result of the registration, left is DRR and right is actual X-ray image.

Fig. 3.Registration of DRR and actual dual X-Ray images.

Fig. 4.Aligned DRR (left) and actual X-Ray iamge (right) after registration.

Fig. 5 shows the result of plotting the minimum value (best ever score) for the error score in each period while performing optimization using Simulated Annealing. From the figure, although it increases locally during the optimization, it can be seen that as the similarity metric is gradually reduced toward the minimum value, the estimation of the pose of the model is performed. In the registration of DRR and dual X-Ray images, about 700 iterations were conducted for the optimization in finding the optimal values of similarity metric. At this time, the number of DRR generation is doubled to about 1,400, because an iteration of similarity check requires two DRRs (each for dual X-ray images). As shown in Table 1, the rendering time for creating a DRR is improved about 4-5 times faster by parallel computing using GPU. If we use CPU computing, it takes about 6 minutes to generate 1,400 DRRs. This time can be reduced to less than 2 minutes with GPU computing. We used GeForce 9800 GT, 512MB (NVIDIA Co., USA) graphic card in the experiment.

Fig. 5.Changes in the error score in the process of registration using Simulated Annealing (x-axis: the number of iterations, y-axis: similarity metric value).

Table 1.Acceleration by parallel computing using GPU [unit: FPS (frame per sec)]

The sample results of non-destructive inspection are shown in Fig. 6. After the registration of DRR and X-ray images, the sample is compared with the master’s DRR. The sample is inspected through image processing including difference and blob operations. The suspicious error areas are displayed after the image processing as shown in Fig. 6.

Fig. 6.Examples of non-destructive inspection of the proposed methods.

 

Conclusion

We proposed a fast non-destructive inspection method using CT scanning of the master and dual X-ray imaging of the sample. The master’s CT scan data are registered to the sample’s the dual X-ray images, and the sample is inspected by the proposed method. The proposed method has advantageous because non-destructive inspection is possible using only dual X-ray images without CT scanning of all sample products. The 2D-3D registration is accelerated by GPU computing. We are planning to quantitatively evaluate the accuracy of the proposed method and investigate the feasibility of using more X-ray images rather than two orthogonal images.