Research on Reconstruction Technology of Biofilm Surface Based on Image Stacking

Abstract

Image stacking technique is one of the key techniques for complex surface reconstruction. The process includes sample collection, image processing, algorithm editing, surface reconstruction, and finally reaching reliable conclusions. Since this experiment is based on laser scanning confocal microscope to collect the original contour information of the sample, it is necessary to briefly introduce the relevant principle and operation method of laser scanning confocal microscope. After that, the original image is collected and processed, and the data is expanded by interpolation method. Meanwhile, several methods of surface reconstruction are listed. After comparing the advantages and disadvantages of each method, one-dimensional interpolation and volume rendering are finally used to reconstruct the 3D model. The experimental results show that the final 3d surface modeling is more consistent with the appearance information of the original samples. At the same time, the algorithm is simple and easy to understand, strong operability, and can meet the requirements of surface reconstruction of different types of samples.

1. INTRODUCTION

1.1 Surface reconstruction

In recent years, in the fields of materials, metal processing and bioengineering, surface reconstruction is often needed to obtain the appearance information that can directly describe the surface or shape of complex objects. With the development of computer-aided design and graphics, surface reconstruction technology has achieved great success both in theory and in application. It has become a powerful tool for discovering and understanding various phenomena in the process of scientific calculation. At the same time, due to the rapid development of science and technology, the data from various aspects of scientific calculation, engineering calculation, measurement and so on are increasing day by day, and the accuracy required is also becoming increasingly accurate, so the study of large-scale scattered data surface reconstruction has become an urgent problem to be solved [1].

Surface reconstruction has been studied for decades at home and abroad, and its classification methods are various. According to the relationship between reconstructed surface and data set, it can be divided into two categories: interpolation method and approximation method. The reconstructed surface obtained by interpolation is completely through the original data points, while the approximation rule is to approximate the original data points by sharding linear surface or other forms of surface. In general, the former method belongs to the field of computer graphics, while the latter belongs to computational geometry. The reconstructed surface can be divided into functional reconstruction and discrete reconstruction. There are two methods for reconstruction of functional surface: automatic reconstruction of topological B spline surface and fitting of discrete points. The most commonly used discrete reconstruction is triangular mesh surface reconstruction [2].

1.2 Laser scanning confocal microscopy

Image stacking technique is one of the key techniques for complex surface reconstruction. The process includes sample collection, image processing, algorithm editing, surface reconstruction, and finally reaching reliable conclusions. Since this experiment is based on laser scanning confocal microscope to collect the original contour information of the sample, it is necessary to briefly introduce the relevant principle and operation method of laser scanning confocal microscope. After that, the original image is collected and processed, and the data is expanded by interpolation method. Meanwhile, several methods of surface reconstruction are listed. After comparing the advantages and disadvantages of each method, one-dimensional interpolation and volume rendering are finally used to reconstruct the 3D model. The experimental results show that the final 3d surface modeling is more consistent with the appearance information of the original samples. At the same time, the algorithm is simple and easy to understand, strong operability, and can meet the requirements of surface reconstruction of different types of samples.

1.3 Comparison of surface reconstruction algorithms at home and abroad

After 3d data scanning, the collected point cloud data should be processed before other further work. At present, the main data processing methods are approximation and interpolation. In recent years, many scholars at home and abroad have done a lot of research on point cloud data denoising, smoothing processing, data simplification and registration, etc., and achieved certain results. However, the simplified point cloud data cannot be directly used in numerical control processing, mold design and other production fields. Currently, the main solution is to form a qualified digital model through NURBS surface fitting or digital modeling. For example, For a given surface model, Zhang Xueqin et al. in China approach the conic axis locally by clustering developable surface, and then conduct regional segmentation of developable surface to construct accurate conic spline surface. Wang Yujian et al. reconstructed the local grid surface by segmenting the compact point cloud data and using NURBS method. After decades of development, great achievements have been made in the study of approximation method, but the method has its own limitations. For example, the approximation method is mainly used to conduct theoretical and practical research on certain characteristic surfaces, such as strip, rotary and gentle surfaces. However, there is no further research on how to adopt more appropriate modeling methods for surfaces with different features. In order to solve the above problems, this paper decides to adopt interpolation method to process point cloud data, and proposes a set of methods for coordinate extraction, interpolation selection and random number generation [4].

2. METHODS

2.1 The experimental process

(1) Laser scanning confocal microscope (LSCM) is used to scan the sample and obtain the whole three-dimensional shape image of the sample. Because the laser scanning confocal microscope is used for layered scanning of samples, a fixed scanning width is required in advance. After scanning the samples layer by layer, a series of slice images of the samples at each scan width were obtained.

(2) MATLAB was used for gray processing of the scanned slice image, and the point with the lowest gray value was obtained. Because the gray value of the low place, the more likely there is a real object, the higher the gray value, the lower the possibility. Then, the horizontal and vertical coordinate values of these discrete points are obtained according to the pre-set scanning width and section area range. The same processing takes place on each slice of the scanned width so that a series of three-dimensional discrete point data can be obtained.

(3) Since a series of discrete points are obtained through the above method, the accuracy of three dimensional images drawn through these discrete points is proportional to the number of discrete points. Therefore, it is necessary to use the mathematical interpolation method to insert a certain number of discrete points between every two discrete points. Therefore, the whole database is expanded, and the more three-dimensional coordinates, the more accurate the drawing.

(4) Using some 3d coordinates extended by interpolation, a volume rendering method is used to complete the drawing.

2.2 The Interpolation

Interpolation is literally similar to inserting one or more numbers between two values, which belongs to the concept of mathematics [5]. Specifically, by extracting the profile points, the shape value points obtained are all discrete points, and the space between them is relatively large. Therefore, how to construct a closed curve from discrete points needs interpolation. In short, the principle of interpolation is to find a continuous function on the basis of many discrete points extracted. It happens that all the points passed by this function contain discrete points, which is called interpolation function. Now that we know the expression of this function, we can easily get the coordinates of other points which are not discrete points by using the mathematical knowledge learned. This method is interpolation. This method can be used not only in function solving, but also in image processing to fill the gap in image transformation [6].

3. POINT CLOUD DATA PROCESSING

Point cloud data processing mainly includes section image processing and three-dimensional image processing, including image graying, discrete point extraction, random point extraction and image drawing. Drawing

3.1 Cross section image processing

The section image processing flow is shown in Fig. 1:

Fig. 1. Flowchart of Section Image Processing.

According to the particularity of the collected pictures, images of different depths will be presented under the confocal laser microscope, and this depth represents the size of the existence of the sample here. Therefore, the first step is to perform grayscale processing on the image, and then find the point with the lowest grayscale value in the entire picture. Theoretically, the lower the grayscale value (the lighter the color, the greater the probability that the sample exists). In the second step, the coordinates of these points are extracted and interpolated according to different interpolation methods to obtain the coordinates of a series of points. The third step uses volume rendering to complete the 3D reconstruction according to the coordinates of these points.

3.2 Extraction of discrete points on images

The original image of the sample is measured by a section and then rotated around the z axis, so the direction of the slice should be z axial. A series of equal spacing planes parallel to the plane xoy are intercepted (z = h_k) forming a set of tangent planes (S_k|k = 1,2...,N)[7]. As long as the distance between the section and the section, the height of the initial section and the bandwidth of the section (optical section) can be determined, a series of optical section images of the 3D model can be obtained [8].

Define bandwidth as: ∆h. According to the above, the bandwidth is determined by the average of the shortest distance between two spatial points adjacent to the axis z on a three-dimensional sample. The points in the region [h-∆h,h] and [h,h+∆h] inside form the upper and lower boundaries of the region, as shown in figure 2.

Fig. 2. Contour Data Point Extraction.

In this neighborhood point set, some points may be above or below. In order to facilitate extraction, I choose one of the above or below projection on the z = h top. Assuming that the total number of discrete points projected above or below is m:

\(P_{i} \mid P_{i}=\left(x_{i}, y_{i}, h\right) i=1,2, \ldots, m\)       (1)

3.3 Ordering of Contour Data

The contour point data extracted by the above method is disordered and can not be used for inter- polation, so the method of ordering contour data is given before interpolation.

In the first step, the first z=h section collected in the experiment is extracted, and the centroid of the closed curve formed by each discrete point on the section is calculated G(x_c,y_c), \(x_{c}=\frac{1}{m} \sum_{i=1}^{m} x_{i}\), \(y_{c}=\frac{1}{m} \sum_{i=1}^{m} y_{i}\).

The origin of the coordinate system of the discrete point (x_i,y_i) on the profile of the object is translated from (x₀,y₀) to the centroid coordinate G(x_c,y_c). After the transformation of the coordinate system, the discrete point with the original coordinate of (x_i,y_i) becomes the coordinate of (x_i-x_c,y_i-y_c). Then let P_i = (x_i-x_c,y_i-y_c), where P_i is about the centroid vector, and then calculate the angle of each discrete point vector relative to the axis in order, which is also called the argument angle

\(\alpha_{i} \alpha_{i}=\arctan \left(\frac{y_{i}-y_{c}}{x_{i}-x_{c}}\right) \quad 0 \leq \alpha_{i} \leq 2 \pi\)       (2)

Among them, 90º, 180º, 270º and other special angles need to be dealt with separately. After getting the amplitude angle of each vector, the most important thing is to arrange each vector according to the size order of the amplitude angle. According to the habit and the convenience of calculation, the elements {α_i} are arranged in ascending order, that is, from small to large, and the order after arrangement is displayed as follows:

\(\left\{\alpha_{i}\right\}, j=1,2, \ldots, m, 0 \leq \alpha_{1}<\alpha_{2}<\ldots,<\alpha_{j}<\ldots<\alpha_{m} \leq 2 \pi\)       (3)

The corresponding contour point sequence is changed {P_j} from {P_i}. Only after the discrete point set is ordered can the function be used spline() to interpolate the closed curve.

3.4 Interpolation method

(1) Recent neighborhood law (nearest) [9]

Among the three key elements, the bandwidth relationship between the cut plane and the section is the largest, the bandwidth is wide, the section thickness is large, the bandwidth is small, and the section thickness is small. Therefore, bandwidth is determined according to the method determined above, which can minimize bandwidth and thickness.

(2) Linear interpolation at gray level(linear)

The principle of this method is to determine the gray value of the unknown section by the gray value of the point on the known section.

(3) Two-dimensional linear interpolation(cubic)

This method is an improvement to the nearest neighborhood method, and the color of the inserted point is obtained by the color synthesis weight of the four points near the inserted point.

(4) Quadric polynomials(v4)

Through known interpolation functions as follows:

\(q(X)=\frac{\sin (\pi x)}{\pi x}\)       (4)

The original function can be obtained by derivation and inverse derivation. The cubic interpolation method is the enhancement of the above two-dimensional linear interpolation. The color value at the point to be inserted is obtained by using the color weighted average of 16 points around the point to be inserted [10].

In the calculation of plane interpolation and space interpolation, the choice of weight has a great influence on the result of interpolation. In the above four interpolation methods, nearest neighbor domain method and the one-dimensional linear interpolation is the most simple, the physical significance of the polynomial interpolation is not clear, to draw some difficult to explain the value of the spline function method is the point value to some limit, by controlling the estimation variance, In the calculation of plane interpolation and space interpolation, the choice of weight has a great influence on the result of interpolation.In the above four interpolation methods, nearest neighbor domain method and the one-dimensional linear interpolation is the most simple, the physical significance of the polynomial interpolation is not clear, to draw some difficult to explain the value of the spline function method is the point value to some limit, by controlling the estimation variance, In conclusion, an inverse distance weighted average method is proposed [11].

The inverse distance weighted average method takes the distance between the estimated point and the measured point as the weight factor. The closer the distance between the estimated point and the measured point, the greater the weight, and vice versa. The weight value is given by the inverse ratio of the distance, then it is called the inverse distance weighted average method, whose expression is,

\(Z_{i}=\sum_{i=1}^{n} \lambda_{i} Z_{0}\)       (5)

Where, z_i is the value of the estimated point, z₀ is the measured value of interpolation elements at point i, λ_i is the weight of various sample points to be used in the budget process, n is the number of measured points participating in interpolation, and the formula to determine the weight is,

\(\lambda_{i}=\frac{d_{i_{0}}^{-P}}{\sum_{i=1}^{n} d^{-P_{i_{0}}}}\)       (6)

Where, P is the power exponent, and \(d_{i_{0}}\) is the distance between the prediction point i and each known sample point. The weight of sample points in the calculation of the predicted point value is affected by the power index P, that is, the distance between the sampling point and the predicted value increases, and the weight of the influence of the sampling point and the predicted point decreases according to the exponential law.

3.5 Summary of Methods and Programming of Surface Reconstruction

The first expression is to keep the original data intact. Although the information can be guaranteed not to be lost, the storage space needed is too large and the memory requirement of the computer is high.

The second expression requires only two characters, 1 or 0, which greatly reduces the amount of data stored, representing objects or backgrounds, respectively, which can reduce the amount of storage. But it also loses a lot of useful information [14].

The third expression is the effective combination of the two, which can not only retain some intermediate data, but also will not greatly increase the burden of memory, will not waste time, so I use this way to deal with it.

When the laser scanning confocal microscope (LSCM) is used to collect the original information, the distance along the z-axis of the scanning device is fixed every time, that is to say, the sections between the sections are fixed. These are three-dimensional arrays F(x,y,z) that can be constructed from these. The array x,y can be determined by the discrete points on the cross-section image, while z can be determined by the preset movement value before measurement.x And y represent the length and width of the tomographic image, and z represent the longitudinal scanning range of each laser [15].

The representation form of volume data is: (x,y,z,v), which (x,y,z) represents the 3D coordinates of the sampling point, and represents the physical properties of the point, which is the key to the expression of volume data. It can represent entity or background, represented by 1 and 0, and other physical attributes, such as color and density, and vector, such as velocity. Gray

4. Procedure of image processing

4.1 The flow chart of the Image processing is shown in figure 3:

Fig. 3. Flowchart of the Image Processing.

4.2 Image processing example

It can be seen from Fig. 8 that when different interpolation methods are selected, the three-dimensional reconstruction results are also different. The "V4" method reconstruction results have rounded edges, close to the actual shape of the sample, and the reconstruction effect is the best. Both the "linear" method and the "cubic" reconstruction method have jagged results, and the reconstruction effect is moderate. The "nearest" reconstruction results have rough edges, a large area peak plane phenomenon, serious distortion, and the worst effect. Therefore, the "v4" method is more suitable for this 3D reconstruction method.

Fig. 4. The Original Sample.

Fig. 5. Gray Processing.

Fig. 6. Gray Classification.

Fig. 7. Contour Extraction.

Fig. 8. 3D reconstruction results.

5. CONCLUSION

This paper studies the core problem of reverse engineering technology, the point cloud data processing of surface reconstruction and the related methods of reverse modeling, which is extremely beneficial to the study of the new field of reverse engineering technology, and provides a new way of thinking for surface reconstruction related workers.

Through binarization and grayscale processing of the original image [16], the highest gray value under each scanned section was finally extracted; According to the scanning width preset by the laser confocal scanning microscope and combined with the local gray value peak extracted before, a series of coordinates of high bright spot are obtained. Finally, the method of one-dimensional interpolation was used to interpolate the discrete points to obtain a complete point cloud database. Ten coordinate values were taken in the experiment.

Different from other surface reconstruction methods, interpolation method is adopted for data processing in this paper. Interpolation method has no specific requirements for the surface shape of the sample itself, and the accuracy of reconstruction results is related to the selection of interpolation methods and the generation of random numbers. The accuracy fitting method of interpolation is relatively low, but it has a wide range of application and provides a new way of thinking for designers.