A Study on an Automatic Multi-Focus System for Cell Observation

Abstract

This study is concerned with the mechanism and structure of an optical microscope and an automatic multi-focus algorithm for automatically selecting sharp images from multiple foci of a cell. To obtain precise cell images quickly, a z-axis actuator with a resolution of $0.1{\mu}m$ was designed to control an optical microscope Moreover, a lighting control system was constructed to select the color and brightness of light that best suit the object being viewed. Cell images are captured by the instrument and the sharpness of each image is determined using Gaussian and Laplacian filters. Next, cubic spline interpolation and peak detection algorithms are applied to automatically find the most vivid points among multiple images of a single object. A cancer cell imaging experiment using propidium iodide staining confirmed that a sharp multipoint image can be obtained using this microscope. The proposed system is expected to save time and effort required to extract suitable cell images and increase the convenience of cell analysis.

1. Introduction

For cell observation, it is necessary to acquire suitable images from several foci, such as the cell surface, nucleus, and stained reagents, according to the observation purpose. We propose an automatic multifocus system that automatically selects sharp images of various foci of a single cell to save significant time and effort in performing cell observation and analysis [1,2]. Unlike an autofocusing system, which finds an existing sharply focused image, we designed a peak detection algorithm to find and select multiple focal points. Additionally, a cubic spline interpolation method is used to shorten the time required to select clear cell images. An optical microscope and lighting control system were constructed for the implementation of our system. It is expected that the proposed system will save significant time and effort in acquiring cell images for various purposes.

2. Automatic Multi-Focus System

2. Automatic Multi-Focus System

Our optical microscope was constructed based on a depth-of-focus scheme and cell images were captured by moving the camera phase in μm-resolution units to implement an automatic multi-focus system. Fig. 1 presents the structure of the optical microscope and Fig. 2 presents a hardware system diagram. Our system is designed to acquire images by selecting lens scale, lighting color, and brightness for cell observation purposes and automatically controlling the z-axis rotary actuator.

Fig. 1. Optical microscope hardware appearance.

Fig. 2. Hardware system diagram.

2.2 Lighting Color and Brightness Control System

When observing cells, the color of the light depend on the reagent that are used for cell staining. Moreover, the brightness of the light should be set considering whether the cell is alive or whether a large number of clustered cells need to be visualized [3–5]. Fig. 3 shows the staining of the surface of mouse macrophages (RAW264.7) with Alexa Fluor 555 (red) and Alexa Fluor 488 (green), staining of the nucleus with DAPI (blue), and images of cell under light of different color. In Fig. 3(a), the green color staining was observed on the cell surface when illuminated with green light, and in Fig. 3(b) the blue reagent was stained on the nucleus when blue light was illuminated. Fig. 3(c) shows that when both red and green lights are used, yellow stain with a mixture of green and red staining can be seen on the cell surface, and only green stain penetrates into the nucleus. Fig. 3(d) shows that when red, green and blue are used at once, yellow with a mixture of green and red staining can be seen on the cell surface, and green and blue stains penetrates the nucleus. It can be seen that appropriate lighting should be used depending on the condition of the cells to be observed.

Fig. 3. Image of mouse cells stained with red, green, and blue reagents: (a) cell image with green lighting, (b)cell image with blue lighting, (c) cell image with red and green lighting, and (d) cell image with red, green and blue lighting.

2.3 Automatic Multi-Focus Algorithm

The proposed automatic multi-focus system converts the RGB values of each pixel of an image acquired using a light microscope into Laplacian values after an initial pass through a Gaussian filter. The Laplacian values for each pixel are summed to represent sharpness numerically and the points with the highest Laplacian sum values are selected using the cubic spline interpolation method and peak detection algorithm. Fig. 4 illustrates the process of executing the automatic multi-focus algorithm. In order to employ this algorithm the preprocessing is carried out, wherein the cells to be observed are placed on the constructed optical microscope the z-axis actuator is moved, and the image is scanned and binarized. Then, the sharpness is converted into a value and extracted for 100,000 images scanned through four stages including Gaussian filter, Laplacian filter, Laplacian mask, and median filter. Next, the cubic spline interpolation method and peak detection algorithm are used to finally select the image with the sharpest focus. In a conventional autofocusing system, it is time consuming to check images at intervals of 0.01 μm when capturing images using a lens with a magnification of 1000× or more. To compensate for these drawbacks, we applied the cubic spline interpolation equation to estimate the Laplacian values without scanning images using a 0.01 μm phase to improve the speed of the system [6,7]. The formula for cubic spline interpolation is shown in Eq. (1) and the explanation about the peak detection algorithm is shown in Fig. 5.

Fig. 4. Automatic multi-focus algorithm flowchart.

\(f(x)=f(x 0)+\frac{f(x 1)-f(x 0)}{x(1)-x(0)}(x-x 0)\)       (1)

Eq. (1) represents the interpolation operation. This interpolation equation can be used to calculate an ݂ሺݔሻ value between ݂ሺݔ0ሻ and ݂ሺݔ1ሻ. In other words, in the cubic spline interpolation function, when Laplacian values from ݂ሺݔ0ሻ to ݂ሺݔ1ሻ are calculated for the phase from ݔሺ0ሻ to ݔሺ1ሻ, the values of ݂ሺݔሻ corresponding to x between ݔሺ0ሻ and ݔሺ1ሻ are also calculated. Therefore, ݔሺ0ሻ and ݔሺ1ሻ perform the interpolation to obtain the x-axis range, and ݂ሺݔ0ሻ and ݂ሺݔ1ሻ perform the interpolation and become the y-axis range in which ݂ሺݔሻ is obtained. Conventional autofocusing systems find one image that is in clear focus. However, when observing a cell, it is necessary to find clear images at various foci, such as the cell surface, cell nucleus, and stained regions. For this purpose, the proposed system incorporates the peak detection algorithm, which automatically selects sharp images in various phases [8,9]. In the process of performing the peak detection algorithm, a smoothing filter removes the noise in the Laplacian transformation values obtained from cubic spline interpolation. Then, baseline correction of the peak detection reference point is performed using a baseline filter and moving average filter. The entire dataset is then shifted to select a peak detection criterion. In this process, shifted waveform analysis (SWA) algorithm is executed. The explanation of this algorithm is shown in detail in Eq. (2). The peak reference is selected through the SWA algorithm. Next, peak points that match the criterion are selected. Fig. 5 illustrates the process of the peak detection algorithm and Fig. 6 presents the graph resulting from the peak detection algorithm.

Fig. 5. Peak detection algorithm flowchart.

Fig. 6. Peak detection result graph.

\(\begin{array}{c} \text { Peak }_{\text {Data}} \text {[phase] }=\left(\text {Org}_{\text {Data}} \text {[phase}\right]-\left(\text {Shift}_{\text {Data}} \text {[phase}\right]-\delta \text { )) }^{\mathbf{2}} \\ \text { (phase }=[0-100,000], \delta=2,000) \end{array}\)       (2)

Fig. 6 contains three peak points after applying the cubic spline interpolation and peak detection algorithms. Eq. (2) represents the SWA operation. This SWA equation can be used to detect a peak. The SWA procedure is shown in Fig 6, where ‘phase’ represents the range of phases for performing peak detection and \(\delta\) represents the interval to shift in the ܱOrg_Data[phase] . [݁The size of the \(\delta\) can be experimented by freely setting the shift interval. The ܱOrg_Data[phase] literally represents the original blue data, and the ܵShift_Data[phase] represents the green data shifted by \(\delta\) . Moreover, Peak_Data[phase] represents the purple data that was finally obtained as a result of formula execution. As a result of Eq. (2), we could obtain the purple peak data of Fig. 6 and identify the three peaks denoted by red dots. Using this method, a clear image of multiple foci of a single cell can be obtained automatically, thereby increasing the convenience and efficiency of the system. Fig. 7 shows the images taken at the three peak points indicated by the red dots in Fig. 6. Fig. 7 presents images obtained from the three peak points after photographing cancer cells (MKN-28) stained with propidium iodide (PI) reagent and performing peak detection. Fig. 7(a) is the outline of the cell, Fig. 7(b) is the PI stained region, and Fig. 7(c) is the cell epidermis. These experimental results demonstrate that sharp images of multiple phases of a single cell can be detected automatically using the peak detection algorithm. From the three images in Fig. 7, it can be seen that the three peak points obtained in Fig. 6 accurately selected sharp images.

Fig. 7. Image of cancer cells (MKN-28) stained with propidium iodide (PI) reagent captured at three peak points: (a) cell image captured at 20,065 μm phase, (b) cell image captured at 47,103 μm phase, and (c) cell image captured at 69,562 μm phase.

Fig. 8. Automatic multi-focus program user interface.

3. Conclusion

In this study, we constructed an optical microscope for accurate cell analysis and implemented an autofocusing algorithm that can automatically acquire the necessary images for cell analysis by focusing on the cell surface, cell nucleus, and dyed reagent. Fig. 8 illustrates the program user interface (UI) for executing the automatic multi-focus algorithm. In order to verify the system performance, we automatically captured three clear images of PI-stained cancer cells focusing on the cell surface, cell outline, and stained regions. In the future, it should be possible to use this method in a reliable automatic skin health diagnosis system by analyzing images of various skin disease cells and constructing a database according to cell health statuses. The proposed system can also be applied to a fluorescence phase-contrast microscope, where the time and effort required to capture cell images can be saved and a significant enhancement in cell analysis convenience is expected.