Classification of Operating State of Screw Decanter using Video-Based Optical Flow and LSTM Classifier

Abstract

Prognostics and health management (PHM) is recently converging throughout the industry, one of the trending issue is to detect abnormal conditions at decanter centrifuge during water treatment facilities. Wastewater treatment operation produces corrosive gas which results failures on attached sensors. This scenario causes frequent sensor replacement and requires highly qualified manager's visual inspection while replacing important parts such as bearings and screws. In this paper, we propose anomaly detection by measuring the vibration of the decanter centrifuge based on the video camera images. Measuring the vibration of the screw decanter by applying the optical flow technique, the amount of movement change of the corresponding pixel is measured and fed into the LST M model. As a result, it is possible to detect the normal/warning/dangerous state based on LSTM classification. In the future work, we aim to gather more abnormal data in order to increase the further accuracy so that it can be utilized in the field of industry.

1. Introduction

Recently, deep learning technology has been converging throughout the industry in the field of machine vision. Among them, the field of Prognostics and health management that monitors the status of mechanical systems and diagnoses failures (PHM, prognostics and health management), is being introduced in various industries[1]. Machine failure causes a change in the quality of the product, and furthermore, the production becomes impossible, and considerable time and economic costs are incurred. In order to maintain the uniform quality of products and improve the equipment operation rate, it is necessary to monitor the machine status and detect abnormal conditions in real time, hence the necessity to take precautionary measures in advance. PHM can efficiently manage machine systems by detecting abnormal conditions and predicting lifespan through machine learning and deep learning algorithms based on various sensor data[2-4].

Following this development, deep learning technology is fused with a screw decanter centrifuge, a type of centrifuge that filters foreign substances and liquids in public wastewater treatment facilities, to detect abnormal conditions of the machine and diagnose system failure in advance. A screw decanter is a type of centrifugal separator, and it has a structure that separates and discharges solid solids and separated liquid by using the difference in specific gravity of the injected sludge while the external bowl and the internal screw rotate separately. For smooth centrifugal separation, the sludge must be slidably transferred to the screw, and various impurities are mixed in the sludge, and these impurities are crushed by the high-speed rotating screw, causing cracks, partial deformation, wear and corrosion damage. If such damage becomes severe, separation and discharge performance is greatly reduced, and vibration of the equipment is greatly generated, which is the main cause of damage to the entire equipment.

The screw decanter diagnoses equipment maintenance and failure according to the degree of wear of screws and bearings. Depending on the degree of wear of the parts, a stronger vibration than in normal operation occurs, and if left unattended, the machine will be fatally damaged and the equipment will stop operating. Therefore, in order to measure the replacement timing of screws and bearings, various sensors are attached to the screw decanter, and the appropriate time for replacement is determined by recording the level and timing of foreign substances inflow. However, in the case of an attached sensor, continuous replacement is required due to the corrosive gas in the wastewater treatment facility, and the vibration and wear of parts of the machine depend on the eyes and experience of the manager to determine maintenance. This causes a lot of loss in inspection costs, and there is a problem in that it is difficult to determine an appropriate maintenance time for the equipment.

In this paper, the vibration pattern is measured by applying the optical flow technology of the screw decanter using camera image information, and a recurrent neural network is used to classify normal/warning/dangerous operation conditions[5]. Prior to the demonstration, a screw decanter failure simulation test bed is built, sensors are attached to bearings and screws to collect normal, danger, and warning data, and a high-speed camera is installed in front of the test bed to collect vibration images. At this time, the OPC UA Server was built to collect data efficiently, and the system was designed to monitor and store data in real time[6].

A study was conducted to classify and determine the operation status of a screw decanter using RNN) and LSTM (Long short-term memory) classification models to create sequence data by applying optical flow[7] to normal/warning/dangerous operation image data thus extracting motion vectors of each pixel[8-9].

2. Methodology

2.1 Optical Flow

Movement of an object or camera in two consecutive frames. The following conditions are assumed for the optical flow; the brightness constancy of a point moved to the next frame must be constant, and spatial smoothness, in which all adjacent points move to the same size[10].

In the first image, a pixel is designated as \(I(x,y,t)\) , and this pixel is moved by \((dx, dy)\) after \(dt\) time in the next image, as in Equation (1). Apply the Taylor series to the right side of Equation (1) to remove the common term. This is the same as Equation (2). After dividing the equation by \(dt\), it can be arranged as in Equation (3), which is called the optical flow equation.

\(I ( x , y , t ) = I ( x + d x , y + d y , t + d t )\)       (1)

\(f _ { x } u + f _ { y } v + f _ { t } = 0\)       (2)

\(f _ { x } = \frac { \partial f } { \partial x } ; f _ { y } = \frac { \partial f } { \partial y } ; u = \frac { \partial x } { \partial t } ; v = \frac { \partial u } { \partial t }\)       (3)

2.2 RNNs and LSTM

RNNs can be described as a class of artificial neural networks where connections between nodes form a directed graph along a temporal sequence, and this allows it to exhibit temporal dynamic behavior[11]. Using loops, RNNs can keep information across time with their design also enabling them to process variable length sequences of input data. However, as the sequence grows longer, it becomes harder for RNN to recognize the interrelationship between the components of the same sequence which are far apart from each other.

The challenge facing the RNNs architecture is the problem of short-term memory because the gradients tend to explode or vanish known ass the “vanish gradient problem” [12-13]. To solve this problem, LSTM is a mechanism that is popularly implemented as it was specifically designed to answer the issue of long-term dependencies remembering information for a long time as its default state[14-15]. The LSTM network built on the basis of the cell state which runs across the network in a straight line with minimal linear interaction, hence allowing easy information flow.

As seen in Fig. 1, while a regular RNN node contains only simple calculation (e.g. tanh operation) which consists of a single neural network layer, the LSTM cell carries 4 neural network layers. These additional neural network layers and linear operators together form structures called gates (i.e.; input gate, forget gate, output gate)[16]. Therefore, the LSTM the network is able to handle longer sequences of sentences better compared to the standard RNN network.

Fig. 1 simple RNN node (a) and an LSTM cell (b)

The illustration shows the RNN node that contains a single neural network layer, in contrast, the LSTM cell carries 4 layers interacting uniquely. The LSTM is elevated by recurrent gates known as forget gates.

3. Implementation and Results

3.1 Data collection and preprocessing

In order to classify the actual screw decanter's operating condition, a failure simulation test bed of a special screw decanter was built and installed to collect data. As shown in Fig. 2, the test bed is a centrifugal separator that meets KS B 6854 standards with 1-3㎥ /hr, standard moisture content of 80±2%, and recovery rate of 95% (based on 1.5% DS) and external ball motor 11kW / screw motor 3.7 / balance degree KS B 0612 of G6.3 or higher / maximum centrifugal effect 3000G was confirmed and manufactured/installed. To collect video information, a camera with a specification capable of recording 1080p/1000fps was installed in front of the test bed to collect video data.

Fig. 2 Status of the Decanter centrifuge

The OPC UA Server was built to efficiently manage and store the sensor data of the test bed's bearings, cylinders, and screw main parts, that is, voltage, vibration and rotation speed data. A web-based OPC UA Client was developed to enable real-time monitoring of the status of the devices using LTE data wireless repeaters[17]. Fig. 3 is the overall system configuration for data collection of the test bed.

Fig. 3 System architecture of data the screw decanter

Normal, warning, and dangerous operation status data were collected in the current test bed. For the dataset, 275 cases of normal operation state, 300 cases of warning state, and 297 cases of dangerous operation state were collected. Data set collection was performed so as not to be affected by illumination by using 4 3,000W lights for high-speed camera shooting.

If the optical flow technique is implemented on the entire image, the amount of computation increases and it is difficult to process in real time, and relatively high vibration occurs. By designating a region of interest (ROI) on a specific part such as a bearing or screw, the optical flow technique was applied to extract data and convert it into a sequence unit. Fig. 4 shows the operating state of the screw decanter and the size of each pixel motion vector per second.

Fig. 4 Part of motion vector of optical flow for ROI

3.2 Simulation

In this paper, training was conducted in the environment of Python 3.6.50 and Tensorflow 2.4.1. OPC UA Server for data collection used Python 3 based open source Free OPC UA.

Time-series learning data was acquired by designating an area of interest in the bearing part sensitive to vibration for normal/warning /dangerous operation condition image data collected from the screw decanter test bed.

For multi-class classification in the existing LSTM classification, the loss function was changed to categorical cross entropy to design a model for classification of three operation states. The overfitting problem was supplemented by specifying the dropout ratio as 0.5.

The accuracy graph is shown in Fig. 5. while For evaluating the LSTM model for the recognised state, we used the confusion matrix shown in Fig. 6. where we obtained an average accuracy of 91.2%.

For performance evaluation, simple Frequency

Fig. 5 Training (blue) and Validation (red) loss

Fig. 6 Confusion Matrix

Analysis, Wavelet Transform Analysis, and Support Vector Machine were used to compare and analyze the operating state of the screw decanter.

The Frequency Analysis classified three operating states with the value of the second peak of the vibration frequency, and the Wavelet Transform Analysis classified the operating states through the same method as the Frequency Analysis through wavelet conversion. The SVM performed two-dimensional linear classification using the value of the motion vector of the optical flow. The comparative analysis results are shown in Table 1 below.

Table 1. Comparative analysis between proposed algorithms and other algorithms

4. Conclusion

Prognostics and health management (PHM), which monitors the status of efficient mechanical systems and diagnoses failures, is converging throughout the industry. This is to optimize productivity by minimizing time and economic costs through immediate response through early detection of machine abnormalities. In this trend, it is attempted to perform abnormality detection to manage the soundness of a screw decanter, a type of centrifugal separator in a wastewater treatment facility. Due to the corrosive gas and harsh environment of the wastewater treatment facility, the sensor attached to the machine needs to be replaced frequently, and the wear measure of important parts such as bearings and screws is being maintained by relying on the eyes of the manager. This results in a large loss cost, and in order to improve this, in this paper, camera image-based screw decanter anomaly detection is performed.

After installing the screw decanter failure simulation test bed, the system was designed to monitor and store other sensor data and image data installed on the top of the test bed in real time through OPC UA Server. In the absence of current failure data, we first tested whether abnormality detection was possible only with normal and stop data. If it detects the normal operation and the stop state, it was judged that abnormal state could also be detected and proceeded with priority. The data collected in the test bed was subjected to a pre-processing process of extracting the pixel motion vector of the bearing part by applying the optical flow technique to create the training data.

The performance was confirmed by training the LSTM multi-class classification model. The accuracy was 91.2% for 600 training data, and it was confirmed that it’s possible to classify the operating state of the screw decanter using the camera image. We plan to secure a dataset through future research and improve the system through model improvement.