Plants Disease Phenotyping using Quinary Patterns as Texture Descriptor

Abstract

Plant diseases are a significant yield and quality constraint for farmers around the world due to their severe impact on agricultural productivity. Such losses can have a substantial impact on the economy which causes a reduction in farmer's income and higher prices for consumers. Further, it may also result in a severe shortage of food ensuing violent hunger and starvation, especially, in less-developed countries where access to disease prevention methods is limited. This research presents an investigation of Directional Local Quinary Patterns (DLQP) as a feature descriptor for plants leaf disease detection and Support Vector Machine (SVM) as a classifier. The DLQP as a feature descriptor is specifically the first time being used for disease detection in horticulture. DLQP provides directional edge information attending the reference pixel with its neighboring pixel value by involving computation of their grey-level difference based on quinary value (-2, -1, 0, 1, 2) in 0°, 45°, 90°, and 135° directions of selected window of plant leaf image. To assess the robustness of DLQP as a texture descriptor we used a research-oriented Plant Village dataset of Tomato plant (3,900 leaf images) comprising of 6 diseased classes, Potato plant (1,526 leaf images) and Apple plant (2,600 leaf images) comprising of 3 diseased classes. The accuracies of 95.6%, 96.2% and 97.8% for the above-mentioned crops, respectively, were achieved which are higher in comparison with classification on the same dataset using other standard feature descriptors like Local Binary Pattern (LBP) and Local Ternary Patterns (LTP). Further, the effectiveness of the proposed method is proven by comparing it with existing algorithms for plant disease phenotyping.

1. Introduction

Food and Agriculture Organization (FAO) of the United Nations predicts 9.1 billion population on earth by 2050. Accordingly, the food production till 2050 must be increased by 70% [1]. Major challenges against increasing food production include: limited agricultural land, crops disease and availability of fresh water. Situation becomes worst due to climate changes that introduce various new types of plant disease, while our knowledge of physiological mechanism that links pathogen infection and presence of disease symptoms remains limited. In developing countries crops disease scouting is still done by visiting fields. It does not seem practically viable to consult an expert every time a plant is infected, secondly, neither farmer nor agriculture consultant can access every plant for problem inspection [2]. Detection of plants diseases thus urges researchers to develop and investigate computer vision-based techniques to ease this arduous task

Considering major dietary source of the antioxidant lycopene and largest source of vegetables tomato and potato remain extremely important crop the world over. These antioxidants are enriched source of several vitamins and minerals including Vitamin C & K1, Potassium and Folate (B9) [3]. Nutritional flavonoids are the major cause of decreased mortality, and apples are the main source of these flavonoids & antioxidants. Only, in United States, in comparison with other fruits involve in antioxidant activities apples are the second highest since they provide 22% of phenolics compounds when compared to other fruits [4,5]. Further, consumption of fruits, vegetables and their related products has been linked to many health benefits regarding cancer prevention, heart disease and skin problems [6,7]. According to a research, [8] fruits and vegetables have a significant shielding effect against a variety of cancer. Being major economic row crops, global sales from tomato, potato and apple exports in 2018 amounted to US$ 21.43 billion [9]. Table 1 lists top countries that exported these crops of highest dollar value during 2018 [10]. Several yield-limiting factors can undermine the value of fruits and vegetables as agricultural commodity. These include abiotic factors like temperature, water, air, light and soil (nutrients) and biotic factors such as pests, insects, diseases and weeds. All these factors are potential threat to profitable production of fruits and vegetables. In this context, we have chosen tomato, potato, and apple as target crops for this study.

Table 1. List of Top Exporting Countries in 2018 for Tomato, Potato and Apple

In this context, this study proposed an automatic and precise plant disease phenotyping system, which employ single channel grey level Directional Local Quinary Patterns (DLQP) [11] as a feature discriptor. Extraction of four directional quinary patterns (two uppers and two lowers) make the proposed approach rotation invariant. Further, due to considering magnitude of the intensity difference between the center reference pixel and neighboring (3x3 window) pixel the proposed method is extremely low imune to noise. Consequenlty, shown improved performance results over other similar local discriptors like Local Binary Pattern (LBP) [12] and Local Ternary Patterns (LTP) [13] for plant disease detection. An added advantage of the proposed method would be reduced computational complexity due to single channel grey level representation of the plant leaf image as a replacement of three color (R, G, and B) channels, details are given in materials and methods section. This paper is structured as follows: a brief review and related work is given in section 2 and in section 3 materials and methods are presented. Result discussion and evaluation of the proposed method are given in section 4. Finally, section 5 derived the conclusion.

2. Related Work

Image processing techniques are applied to diagnosis of plant diseases in many crops such as wheat, maize, cotton, potato, tomato and apple etc. [14] proposed a pattern recognition-based system for identification and classification of disease infected cotton leaves. In this study researchers targeted the common diseases of cotton like Bacterial Blight, Myrothecium and Alternaria. Image segmentation was done using active contour model and Hus moments were extracted for feature extraction. The study claims 85% accuracy. In [15] authors proposed an innovative cucumber's disease recognition method which is based on three sequential steps. First step is segmenting diseased image using K-means algorithm, second is extracting shape and color features from segmented images. The third and last step is to classify diseased images using Sparse Representation (SR). Study claims that SR reduced computations and improved the performance in terms of recognition. Proposed technique recognized seven cucumber diseases with 85.7% recognition rate.

The study [16] proposed a plant disease recognition method using leaf images. Initially diseased spot on leaf is segmented for extraction of feature vector. Then K-Nearest Neighbor (KNN) is applied to show proposed method's efficiency dealing with five kinds of maize diseases. Presented results justified the proposed technique for high disease recognition rates. [17] presents an extension of Neural Network (NN) model based on combination of three-color components titled as Three-Channel Convolutional Neural Networks (TCCNN). This approach has been successful in automatic extraction of high-level discriminant features from colored diseased leaf image resulted in higher efficiency in comparison with many cutting-edge systems of vegetable leaf disease recognition. In [18] authors present an image segmentation technique that can be used for automatic recognition and classification of infected plant leaves. The proposed technique was tested and verified on various fruits and vegetables including banana, beans, jackfruit, lemon, mango, potato, tomato, and sapota. Study claims less computational efforts with high efficiency. Accuracy rate of proposed model can be improved by using Artificial Neural Networks (ANN), Bayes Classifier, Fuzzy Logic and Hybrid algorithms.

In [19] author presented a Support Vector Machine (SVM) based approach for the detection and identification of unhealthy tomato leaves. Image processing technique has been major part of system using Gray-Level Co-occurrence Matrix (GLCM). In this study total 800 mages (healthy and infected) were used for training and testing while cross validation was done using N-fold technique. High classification rate is reported using linear kernel function. In [20] an intensity-based feature extraction approach was presented for tomato plant disease recognition. During feature extraction phase, a total of ten intensity based statistical features were computed for each plant disease. These extracted features were used for classification using decision tree which generated 78% recognition accuracy rate. The study used images of six different types of plant disease infecting the leaves and stem.

Another study [21] presented a computer vision based system for detection of mycotic infections on tomato plant leaves. The proposed system is based on characterization of tomato leaves by Moving Picture Experts Group (MPEG) standard color descriptors which include Color Structure, Color Layout and Scalable Color Descriptors. The reported results show that calable Color Descriptors outperforms all other reported descriptors. In [22] author proposed a potato late blight prediction model using Support Vector Regression (SVR) named as BLITE-SVR. The study used 13 kinds of abiotic factors mainly including temperature, humidity and evaporation which showed very high correlation to the first date of occurrence of late blight on potato leaves. The proposed model claimed 64% occurrence prediction.

Regardless of significant efforts, existing plant disease identification methods are not adequate or enough due to inherent discrepancies. It includes the detection which is not limited to: (1) the leaf image features extracted from diseased crops which are sensitive to illumination, direction and scaling of the images; (2) all selected features are playing important role in classification though they are just representing the algorithm strength. Further, it is difficult to determine which features are optimum and robust for disease recognition. To improve the accuracy and responsiveness of the diagnosis results, many techniques based on different features (i.e. color, texture, and shape) to classify healthy and diseased leaves have been proposed. Few of them have yielded good performance under controlled conditions whereas achieving accuracy in uncontrolled conditions remains a challenge. As far as the recently emerged Deep Learning (DL) techniques are concerend the image processing techniques are still prefered because of inadequacy in mechanism for learning abstractions through explicit verbal definitions. Moroever, DL techniques works best when there are thousands, millions or even billions of training examples [23]. Therefore, DL techniques are not quite fit yet for plant leaf disease classification due to lack of such infinite data and computational resources.

We used images from Plant-Village dataset published online at website www.Plant-Village.org [24], which includes images of several crops. To test and evaluate our proposed meathod, tomato, potato, and apple are choosen as the target crops because these are considered major economic row crops and secondly, due to high consumption rate in daily life. This study used 3900 tomato plant leaf images comprising of 6 diseased classes, 1,526 potato plant leaf images comprising of 3 disease classes and 2,600 apple leaf images comprising of 3 disease classes. Fig. 1. shows infected leaf images of target crops.

Fig. 1. Sample images from Plant Village dataset of Tomato [a-g], Apple [h-k], and Potato [l-o]

3. Materials and methods

3.1 Local Binary Patterns

Local Binary Pattern (LBP) [12] is a texture-based pattern operator. Efficiency with simplicity is main characteristic of LBP. The main advantage offered by LBP is to label neighboring pixel values with reference to their center pixel from selected window and computing result as binary numbers. LBP has been applied to diverse applications, more specifically dealing in face detection. Main contribution of local binary patterns as an operator has been the robustness to monotonic gray-scale changes like illumination variations. LBP represents the window's centered reference pixel n_c and n_p neighbor's gray values which are the total neighbors (N) of that window and its radius R of the neighborhood as shown in Eq. (1). Here value of function f1 depends upon x as defined by Eq. (2).

\(L B P_{Y, R}=\sum_{Y=1}^{N} 2^{Y-1} * f_{1}\left(n_{p}-n_{c}\right)\)       (1)

\(f_{1}(x)=\left\{\begin{array}{ll} 1 & x \geq 0 \\ 0 & \text { otherwise } \end{array}\right.\)       (2)

LBP in deals with texture classification as a discriminative feature descriptor and shows its invariance for monotonic gray level transformation and high resistance to lighting effects. LBP is extremely noise sensitive due to center reference pixel thresholding for its neighbor gray levels as shown in Fig. 2.

Fig. 2. Calculation of LBP value from its neighbourhood

3.2 Local Ternary Patterns

Local ternary pattern (LTP) [13] as a feature descriptor uses 3-valued code, gray-levels in region with range of width ± r_t around p_c central pixel value are quantized to zero, ones above this are quantized to +1 and ones below it to -1, i.e., the indicator S'(g) is replaced with a 3-valued function. Here r_t is threshold specified by the user so LTP codes are more resistant to noise. As defined by Eq. (3).

\(S^{\prime}\left(g, p_{c}, r_{t}\right)=\left\{\begin{array}{cl} 1, & g \geq p_{c}+r_{t} \\ 0, & \mid g-p_{c} k r_{t} \\ -1, & g \leq p_{c}-r_{t} \end{array}\right.\)       (3)

Where g = (p_i - p_c); here p_i is the neighboring pixel values, and p_c central pixel value

3.3 Directional Local Quinary Pattern (DLQP)

DLQP [11] features have been computed for single channel gray scale images and Directional Binary Code (DBC) is used to encode directional edge information [25]. For a gray scale image (I_mg), first-order derivatives (I'_mg(d_k)), occur first, along 0^o, 45°, 90°, and 135° directions. Detailed calculations are shown in Fig. 3. For (Img), first order derivatives are calculated for each direction used in Eq. (4), here d denotes the direction (0°, 45°, 90°, and 135°) and d_c is the intensity of the center pixel in a respective direction. The local quinary values are calculated using Eq. (5), where directional edges are calculated using Eq. (6).

\(I_{m g}^{\prime}\left(d_{k}\right)=I_{m g}\left(d_{c}\right)-I_{m g}\left(d_{k}\right) ; \quad k=1,2,3,4\)       (4)

\(f_{2}\left(p_{c}, \tau_{2}, \tau_{1}\right)=\left\{\begin{array}{rl} +2, & p_{c} \geq \tau_{2} \\ +1, & \tau_{1} \leq p_{c}>\tau_{2} \\ 0, & -\tau_{1}>p_{c}<\tau_{1} \\ -1, & \tau_{2}<p_{c} \leq-\tau_{1} \\ -2, & p_{c} \leq-\tau_{2} \end{array} \mid\right.\)       (5)

Fig. 3. Calculation of Zero Degree DLQP

Where τ₁ and τ₂ are the upper and lower threshold parameters, respectively. The N^th order DLQP for Eq. (6) is defined using Eq. (7).

\(\hat{I}_{\alpha}^{D L Q P}\left(d_{c}\right)=f_{3}\left(I_{m g}^{\prime}\left(d_{l}\right)\right) ; l=(1+\alpha / 45)^{\prime \prime} \alpha=0^{0}, 45^{0}, 90^{0} \text { and } 135^{0}\)       (6)

\(\left.D L Q P^{N}\left(I_{m g}\left(d_{c}\right)\right)\right|_{\alpha}=\{ I_{\alpha}^{D L Q P}\left.\left(d_{c}\right) ; I_{\alpha}^{D L Q P}\left(d_{1}\right) ; \ldots \ldots \ldots \ldots \ldots . . . I_{\alpha}^{D L Q P}\left(d_{8}\right)\right\}\)       (7)

Complete description of DLQPs is as follows. Steps for the calculation of quinary pattern in 0 degree are explained in Fig. 3. After applying the 0-degree first order derivative we retrieved -1, -1, 3, -2, 1, 3, 2, -2, 0 representing texture information. Further, derivatives are coded into quinary pattern 0, 0, 2, -2, 2, 2, 2, -2, 0 using upper and lower thresholds (τ₁ =2 and τ₂ =1). These patterns are further coded in two upper and two lower binary patterns. First upper pattern is attained by replacing 0 for 0, 1, -1, -2 and retaining 2 by 1. Similarly, other upper pattern is attained by retaining 1 by 1 and replacing 0 for other values in said pattern. Likewise, same procedure is applied to the two lower patterns by retaining -1 and -2 by 1 accordingly. Now quinary pattern was successfully converted into four binary patterns, two upper and two lower. Finally, weights are applied to each pattern to get four decimal numbers pointed to DLQP. Constructing DLQP histogram for each pattern using 20 bins and shaping it in final feature vector ready to be used by SVM for classification. The obtained patterns for each directions is shown in Fig. 4, where Fig. 5 illustrates a complete flow diagram of the proposed system.

Fig. 4. DLQP patterns for each directions

Fig. 5. Flow diagram of Proposed System

Dataset images are converted to grey scale (having value from 0 to 255) with dimension of 256 x 256 pixels each. Quality of the image is enhanced using histogram equalization technique as preprocessing step. Subsequently, the background of image is removed, and Region of Interest (ROI) is extracted by segmenting and targeting the diseased portion. Now the system is ready to extract features on this image by applying DLQP descriptor. After getting the final feature vector and in classification phase all three categories, namely tomato, potato, and apple, are grouped in different disease classes. In case of tomato, classes are Bacterial Spot, Early Blight, Late Blight, Septoria, Target Spot and YLCV. For potato, classes are Early Blight, Late Blight and Ring Rot, and for apple SCAB, Cedar Rust and Black Rot are taken. Dataset images were in JPEG format and converted into grayscale files.

4. Results and Discussion

Performance of the proposed system is evaluated in terms of Accuracy, Precision, Recall, F-1 Score and Error Rate. Here, specificity as precision rate determines the ability for correct classification of plant leaf disease against total number of detected disease classes. Precision can be calculated using Eq. (8), the capability of the disease phenotyping system for the association of model to its concerned class is computed using Eq. (9). To evaluate the accuracy of classification performed by the proposed technique F-1 Score is also computed, which imparts the balance or weighted average representation between the precision and the recall. F-1 Score is calculated using Eq. (10). Another performance metric adopted for the performance assessment is the accuracy measure, and is calculated using Eq. (11). Error rate is also calculated using Eq. (12).

\(\text { Precision Rate }=\frac{\text { True Positive }}{\text { True Positive }+\text { False Positive }}\)       (8)

\(\text { Recall Rate }=\frac{\text { True Positive }}{\text { True Positive }+\text { False Negative }}\)       (9)

\(F-1 \text { Score }=2 * \frac{\left(\text { Precision }^{*} \text { Recall }\right)}{\text { Precision }+\text { Recall }}\)       (10)

\(\text { Accuracy Rate }=\frac{\text { True Positive }+\text { True Negative }}{\text { Total Positive }+\text { Total Negative }}\)       (11)

\(\text { Error Rate }=\frac{\text { False Positive }+\text { False Negative }}{\text { Total Positive }+\text { Total Negative }}\)       (12)

For disease classification Support Vector Machine (SVM) is used as a classifier. After getting the training set, the random selection of 70% of the data for training and 30% for testing purposes has been carried out. Fig. 6 reflects the comparison results for accuracies, where DLQP achieve 95.6% for Tomato Plants while 97.8% on Apple and 96.2% on Potato plants using the Medium Gaussian kernel for SVM.

Fig. 6. Recognition rates using different local patterns

To reveal the capabilities of DLQP as a reliable image descriptor, system is evaluated by performing its validation over multi-category problem. Classification results over the multiclass problem confirm the robustness of DLQPs on plant disease phenotyping as the best feature descriptor. The high correlation between the classes makes the instance recognition problem even more challenging. Due to high correlation, miss-association rate is high as it increases classification biasness due to smaller instances in skewed class. The diversity of content in training samples also favors incorrect associations. Class imbalance raises a series of issues like class overlapping and small disjuncts that further increase the difficulty of classifier's learning. However, optimal feature basis increases feature separability amongst classes and, hence, ensures improved classification results. Precision graph for different diseases of apple are shown in Fig. 7, where precision graph for potato and tomato disease classes are shown using Fig. 8 and Fig. 9 respectivly.

Fig. 7. Precision Graphs for Apple Disease Classes

Fig. 8. Precision Graphs for Potato Disease Classes

Fig. 9. Precision Graphs for Tomato Disease Classes

F1-score, recall, and false discovery rates are listed in Table 2. Results for selected evaluation parameters against relative schemes clearly signify the reliability of DLQP as feature descriptor. ROC curves using DLQP based features on Tomato images is shown in Fig. 10, while for Potato and Apple images ROC is shown in Fig. 11 and Fig. 12 resepctivly.

Table 2. Multiclass Evaluation Results for Tomato, Potato and Apple Disease Classes

Fig. 10. RoC Curve using DLQPs based Features on Tomato images

Fig. 11. RoC Curve using DLQPs based Features on Potato images

Fig. 12. RoC Curve using DLQPs based Features on Apple images

Comparison of proposed technique with existing techniques of crop diseases identification is presented in Table 3. It can be witnessed that DLQP as feature descriptor outperformed other descriptors, including LTP and LBP, by achieving highest accuracies.

Table 3. Multiclass Evaluation Results for Tomato, Potato and Apple Disease Classes

6. Conclusion

This study proposed an automatic approach for image-based phenotyping of plant disease using Directional Local Quinary Patterns (DLQP) as feature descriptor and Support Vector Machine (SVM) as a classifier. The proposed DLQP based system is specifically used for agricultural applications. Six tomato leaf diseases, three potato, and three apple leaf diseases are taken for experimentation. For each disease we performed a classification process and compared the individual performance of DLQP, LTP and LBP as feature descriptors. It is found that proposed DLQP texture feature descriptor improves the performance for plant disease phenotyping. The maximum detection efficiencies of 97.8% for apple, 95.6% for tomato, and 96.2% for potato are achieved using DLQP and Medium Gaussian kernel for SVM. Also, a comprehensive comparison shows that the proposed method performs significantly well as compared to existing methods. The proposed system provides promising results for plant disease phenotyping but there is a scope for improvement by using combination of other shape and color-based feature descriptors with DLQP. Moreover, use of ensemble methods as classifier may improve the results.