Use of Support Vector Machines and Artificial Neural Network Methods in Variety Improvement Studies: Potato Example Volume 6 - Issue 1

Mehmet Metin Ozguven¹*, Gungor Yilmaz², Kemal Adem³ and Cemil Kozkurt⁴

• ¹Tokat Gaziosmanpaşa University Department of Biosystems Engineering, Turkey
• ²Tokat Gaziosmanpaşa University Department of Field Crops, Turkey
• ³Tokat Gaziosmanpasa University, Department of Informatics, Turkey
• ⁴Tokat Gaziosmanpaşa University, Department of Mechanical Engineering, Turkey

Received: January 12, 2019;   Published: January 22, 2019

Corresponding author: Mehmet Metin Özgüven, Tokat Gaziosmanpaşa University Department of Biosystems Engineering, Tokat, Turkey

DOI: 10.32474/CIACR.2019.06.000229

Abstract PDF

Abstract

In order to make a contribution to the early generation selections in potato varieties through a classification, the MLPNN and SVM data mining methods were applied to the data set created by considering the selection criteria based on the macroscopic observations and measurements, performed to identify clones that are ineligible and to be eliminated through negative selection from the clones developed in line with the potato variety breeding program, initiated by hybrid combinations in this study. Data set used in the study consists of clones in a study conducted in 2016 as part of the project no. TUBITAK-TOVAG 113O928. A total of 703 potato clones from 12 hybrid combinations were used in the study. In order to identify the clones to be selected, two different models were created by using three attributes (tuber yield, number of tubers and average tuber weight) for each clone, and two different models were created by using two attributes (eyes depth and eyes pit depth) of each clone in order to identify the clones to be eliminated. Experiments were carried out by comparing the sensitivity, specificity and accuracy ratios for each model by using the generated dataset as input to the MLPNN and SVM classifiers. As a result of the experimental studies, the highest success was achieved in 2-class models and it was determined that MLPNN classifier is more successful in these models. With this study, it was put forth that data mining methods can be used for early generation selection in cultivar improvement studies.

Keywords: Potato; Clonal Selection; Breeding; Agricultural Information Technologies; Multilayer Perceptron Neural Network; Support Vector Machine

Introduction

Today, intelligent machines and production systems that control machines have begun to take over traditional production methods. After the mechanization, automation, control and information technology use during the development period of agricultural production, now the data mining methods are an important research area for agricultural science and a promising tool for problems in the agricultural sector that need solutions and improvement [1-12]. Through the identification of plant, weed, disease and soil pattern classes using data mining methods in agriculture, numerous studies have been conducted such as fertility, biomass, chlorophyll, breeding environment and agricultural drought forecasts, various analyzes, modeling and simulations, mineral elements, toxic elements, organic and inorganic pollution and evaluation of biological markers. The summary data of some studies are presented in Table 1.

The potato is one of the plants that spread naturally in the Andes of Peru [13]. It has been brought through Russia and the Caucasus to Turkey, and it has been cultivated in the highland climate of eastern Black Sea region and Erzurum region for the first time in Turkey [14]. In today’s conditions, potatoes can be grown in almost every region of Turkey. Its yearly production is 376 million tons in 19 million hectares of land in the world [15] and 4.8 million tons in the 143 thousand hectares of land in Turkey [16]. Potato is a widely used plant in human nutrition and industry because of its carbohydrate, protein, mineral and vitamin content. Therefore, there are 100-400 commercially registered potato varieties in the countries where intensive potato farming is carried out. Although Turkey is among the most important potato producing countries, the number of native varieties is quite less. Therefore, Turkey is dependent on foreign resources in terms of basic seed varieties [17,18]. Various enhancement works are carried out for the solution of this problem. In potato, variety improvement studies usually start with crossing and continue with clonal selection [19,20].

The main purpose of potato improvement is to increase yield and quality. Besides, it is required to develop varieties suitable for market demands, resistant to temperature and drought as well as resistant to various diseases and pests and storage [21]. When we look at potato improvement programs, we see four fundamental stages. These include: crossing, early generation selection studies, late generation selection studies and advanced generation selection studies based on regional experiments. This period can take about 10-12 years [19,22]. This study aims to contribute to early generation selection by using data mining methods and taking into consideration the selection criteria based on macroscopic observation and measurements, from the clones in a variation set created through crossing studies for the development of native potato varieties. It was aimed to propose a new method, alternative to the classical early generation selection method, by developing a model with Support Vector Machine (SVM) and Artificial Neural Networks (ANN) from the data sets and performing a classification with this model in accordance with the selection criteria.

Materials and Methods

Part of the clones grown under the polycarbonic greenhouse conditions in the Gaziosmanpaşa University Faculty of Agriculture, Department of Field Crops, in 2016, within the scope of the TUBITAK-TOVAG 113O928 project, were used as the study material in this research, and a total of 703 potato clones from 12 hybrid combinations were used in the study. These clones were grown in 26 liter pots having a mixture of peat (2/3) and perlite (1/3) in a polycarbonate greenhouse environment. The ambient temperature was set 22°C during the day, and 15-16°C at night. Necessary maintenance procedures such as fertilization, irrigation and insecticide use were also performed without interruption.

After harvest, the characteristics such as tuber yield per plant (g/plant), average tuber weight (g), number of tubers per plant (number), and depths of eyes and eyes tips in harvested tubers were determined. At this stage of the selection process, positive selection of the clones having marketing and yield-related characteristics and the characters were performed and the clones that did not possess these criteria were eliminated by negative selection. The descriptive statistics of the potato clone’s data set are shown in Table 2.

In this research, experimental studies were carried out regarding the classification of favorable clones and the clones to be eliminated by negative selection from the resulting varieties obtained as a result of the crossing studies. In order to identify the selected clones, two different models were created by using three attributes, “tuber yield (TY), tuber count (TC) and average tuber weight (ATW)” for each clone. Model-1 consists of 2 classes, including selected (1) and other (0) classes. Model-2 consists of 4 classes, including average tuber weight (3), tuber count (2), tuber yield (1) and other (0) class. Two different models were created by using two attributes (eyes depth [ED] and eyes pit depth [EPD]) of each clone in order to identify the clones to be eliminated. Model-3 consists of 2 classes: needs to be eliminated class (1) and the other class (0). Model-4 consists of 4 classes, which are tuber shape (TS) (3), stem scar depth (2), eyes depth (1) classes, which needs to be eliminated, and the other (0) class. The sensitivity, specificity and accuracy ratios for each model were compared by using the generated dataset as input to the Multilayer Perceptron Neural Network (MLPNN) and SVM classifiers. In the MLPNN model, 10 neurons were used in the hidden layer, sigmoid function was used for activation, and back propagation algorithm was used for training. In this method, a randomly selected cross validation method is used that allows the model to be tested neutrally. In the study, the data sets were divided into 40% training, 20% validity and 40% test data, and the average success rate was determined. Gauss function was used as the kernel function in SVM classification.

Support Vector Machine

Since classification is the most basic operation in the estimation part of the data mining, an important part of the problems is encountered in this step. One of the various algorithms used for accurate classification of data sets in the data mining is SVM. Proposed for the first time in 1995 [23], SVM is a supervised learning model in data mining used for classifying binary or multiple data sets of linear separable or linear non-separable type. Since the SVM approaches the classification problem as a quadratic optimization problem, it saves the number of operations in the training process and provides a considerable speed advantage compared to other algorithms [24]. Thus, SVM is successful in highvolume data sets, as well as in high-dimensional problems with few data [25]. While hyperplane based support vectors are constructed for linear-separable data, kernel functions and support vectors are formed for linearly non-separable data. These functions are usually polynomial and Gaussian kernels [26]. SVM is used in many areas of classification problem, such as image processing, financial estimation, biological species detection, medical examination [27- 31]. The hyperplane in Figure 1 is determined by the decision function estimated for the linear-separable data using the SVM method.

Mathematical equations for the support vectors in SVM are given in Equation 1 and Equation 2 to be used in a binary classification problem that can be differentiated linearly.

Where y is the class label, w is the weight vector, and b is the approximation value. The minimizing process of the w value required for increasing the optimum plane is given in Equation 3 [23].

Equation 3 gives the following:

The solution of Equation 4 with Lagrange equations gives Equation 5.

The decision function of the support vector machine for a twoclass problem is given in Equation 6 [32].

Artificial Neural Networks

Another model with superior performance ratios used for classification estimation in data mining is known as MLPNN. The MLPNN model consists of an input layer, a hidden layer and an output layer. There are neurons in the hidden layer that contain nonlinear activation function. Figure 2 shows a sample neuron used in MLPNN. All the i1, i2, ..., in-1, in dimensions of the data are multiplied by weights w1, w2, ..., wn-1, wn respectively before reaching the neuron, and the result is primarily collected in the linear processing unit. The output of the linear processing unit is passed to the output layer through the activation function in the non-linear processing unit [33]. Operations in the hidden layer including linear and non-linear processing units are given in Equation 7 and Equation 8 [33].

The function f given in Equation 8 is the activation function of the non-linear processing unit. Sigmoid, hyperbolic tangent and step functions are frequently used activation functions [34].

Different types of backpropagation algorithm are used to determine the weights between the MLPNN neurons [35]. The backpropagation algorithm equations are given in equations 9, 10 and 11.

The error between the output value y’ and the actual value y is used for updating weights. In the updating process, usually the slope minimization method is used that provides a convergent approach to the goal [36]. Updated value of each weight (Δw) is found by distributing the calculated error energy, inversely proportional to the present w weights, to all the weights coming to the corresponding neuron [37]. The process is repeated for every point of the data and the average of the solutions found through the parameter η can be calculated.

Results and Discussion

Four different models were created for classification of selected clones and clones to be eliminated with the attribute data set obtained from potato clones. The confusion matrices obtained using MLPNN and SVM classifiers for Model-1, Model-2, Model-3, Model-4 are given in Tables 3 & 4. In the experiments for classification of selected potato clones performed for Model-1, 110 out of total 669 clones were classified as clone (1) class, and the remaining 559 were classified as the other (0) class. As seen in the confusion matrix obtained with MLPNN classifier in Table 3, 39 clones were misclassified. As seen in the confusion matrix obtained with SVM classifier in Table 4, 46 clones were misclassified. As a result of the experimental studies, MLPNN classifier was found to be more successful than SVM for Model-1. In the experiments for the classification of selected potato clones performed for Model-2, 72 out of 696 clones were selected in class 3 (mean tuber weight), 34 were as class 2 (number of tubers), 31 as class (tuber yield), and 559 were in the other class. As can be seen in the confusion matrix resulting from the MLPNN classifier in Table 3, 22 out of the 72 clones selected for the actual average tuber weight, 13 out of the 34 clones selected in terms of tuber count, 21 out of 34 clones selected in terms tuber yield, 39 out of 559 clones in the other class were misclassified (95 misclassified clones in total). As can be seen in the confusion matrix resulting from the SVM classifier in Table 4, 18 out of the 72 clones selected for the actual average tuber weight, 8 out of the 34 clones selected in terms of tuber count, 16 out of 31 clones selected in terms tuber yield, and 38 out of 559 clones in the other class were misclassified (80 misclassified clones in total).

As a result of the experimental studies, it was seen that SVM classifier is more successful than MLPNN for Model-2. In the experiments for the selection of potato clones to be eliminated for Model-3, 82 out of 703 clones were class (1) clones that must be eliminated through negative selection, and 621 were in the other (0) class, which is the positive selection class. As seen in the confusion matrix obtained with MLPNN classifier in Table 3, 22 clones were misclassified. As seen in the confusion matrix obtained with SVM classifier in Table 4, 25 clones were misclassified. As a result of the experimental studies, MLPNN classifier was found to be more successful than SVM for Model-3. For the selection of potato clones that should be eliminated by negative selection for model-4, 21 out of 703 clones were class 3 (tuber shape), 39 were class 2 (stem scar depth), 22 were class 1 (eyes depth) clones, which were among the clones to be eliminated, and 621 were in the other class, which is positive selection class. As shown in the confusion matrix resulting from the MLPNN classifier in Table 3, 12 out of 21 clones, which needs to be eliminated in terms of tuber shape, 29 out of 39 clones, which needs to be eliminated in terms of eyes pit depth, 12 out of 22 clones, which needs to be eliminated in terms of eyes depth, and 20 out of 621 clones, which were actually in the other class, were found to be misclassified (73 clones were misclassified in total). As seen in the confusion matrix resulting from the SVM classifier in Table 4, 8 out of 21 clones, which needs to be eliminated in terms of tuber shape, 23 out of 39 clones, which needs to be eliminated in terms of eyes pit depth, 7 out of 22 clones, which needs to be eliminated in terms of eyes depth, and 24 out of 621 clones, which were actually in the other class, were found to be misclassified (62 clones were misclassified in total). As a result of the experimental studies, it was observed that SVM classifier was more successful for Model-4. The sensitivity, specificity, and accuracy values obtained with the classifiers used for 4 different models are given in Table 5.

As shown in Table 5, it is seen that MLPNN was successful in Model 1 and 3, where there were only 2 classes, whereas SVM method was found to be more successful in Model 2 and Model 4, which have 4 classes. The correct classification ratios obtained in experiments on clones to be eliminated were higher than the success rates achieved in experiments on clone selection. This suggests that the selected attributes in the model regarding the clones to be eliminated affect the classification process better. The reason that sensitivity success rate was lower compared to specificity success rate in all models is believed to be due to the fact that the positive/ negative selection clone counts were not sufficient among all the clones. This leads to the necessity of model selection according to number of clones to be evaluated. The number of clones examined in this study was 703, and MLPNN method was found to be more successful in 2 class models, whereas SVM model was better in 4 class models. In variety improvement studies, selection is used as a classical method of improvement. In this method, selection of the genotypes in accordance with the criteria specified is called “positive selection”, and elimination of the inappropriate ones is called “negative selection”. In this research that was conducted with 703 potato clones, which were from an improvement program carried out with selection after crossing, clones were selected and eliminated in accordance with marketable tuber features, such as tuber shape, skin smoothness, eyes and eyes pit depth, in addition to features such as tuber count per plant and tuber weight.

In this study, a new early generation selection method is proposed using the models formed by applying SVM and ANN data mining methods to the selection criteria data sets that have been created as the result of crossing studies for the improvement of native potato varieties. It was determined that results obtained by classification of the data sets of the clones through the models (Model-1, Model-2, Model-3 and Model-4) created using data mining approach are very similar to the results obtained by applying classical methods on the same clones and data sets. The MLPNN method was found to be more successful in the 2-class models with positive and negative selections (Model-1 and Model-3), and the success rates were found as 94.2% and 96.9% respectively. In the 4-class models (Model-2 and Model-4), the SVM method was found to be more successful and the success rates were found to be 88.5% and 91.2%, respectively. Based on these results, it was concluded that data mining approaches can be used, in addition to classical methods, when deciding positive or negative selections of the clones in the early selection stages of potato improvement. And, it was decided to continue in future studies by taking other criteria into account with the selected clones in the next generation. Furthermore, it is anticipated that the success rates can be increased by considering different data mining methods, increased number of clones and different attributes in future studies.

Acknowledgment

In this research, some of the data from the TUBITAK-TOVAG 113O928 project were used.

References

1. Camargo A, Smith JS (2009) An image-processing based algorithm to automatically identify plant disease visual symptoms. Biosystems Engineering 102(1): 9-21.
2. Ribeiro A, Ranz J, Burgos-Artizzu XP, Pajares G, Sanchez del Arco MJ, et al. (2011) An image segmentation based on a genetic algorithm for determining soil coverage by crop residues. Sensors 11(6): 6480-6492.
3. Guerrero JM, Pajares G, Montalvo M, Romeo J, Guijarro M (2012) Support vector machines for crop/weeds identification in maize fields. Expert Systems with Applications. 39(12): 11149–11155.
4. Bhange M, Hingoliwala HA (2015) Smart farming: Pomegranate disease detection using image processing. Procedia Computer Science. 58: 280- 288.
5. Taghavifar H, Mardeni A, Hosseinloo AH (2015) Appraisal of artificial neural network-genetic algorithm based model for prediction of the power provided by the agricultural tractors. Energy 93: 1704-1710.
6. Vijaya Lakshmi B, Mohan V (2016) Kernel-Based PSO and FRVM: An automatic plant leaf type detection using texture, shape, and color features. Computers and Electronics in Agriculture 125(2016): 99-112.
7. Nguyen-ky T, Mushtaq S, Loch A, Reardon-Smith K, An-Vo DA, et al. (2017) Predicting water allocation trade prices using a hybrid artificial neural network bayesian modelling approach. Journal of Hydrology 11: 049.
8. Beucher A, Møller AB, Greve MH (2017) Artificial neural networks and decision tree classification for predicting soil drainage classes in Denmark. Geoderma, 11.004.
9. Liu ZW, Liang FN, Liu YZ (2018) Artificial neural network modeling of biosorption process using agricultural wastes in a rotating packed bed. Applied Thermal Engineering 140: 95-101.
10. Akbarzadeh S, Paap A, Ahderom S, Apopei B, Alameh K (2018) Plant discrimination by support vector machine classifier based on spectral reflectance. Computers and Electronics in Agriculture 148: 250-258.
11. Niel S, Jesús F, Díaz R, Mendoza Y, Notte G, et al. (2018) Beehives bio monitor pesticides in agroecosystems: simple chemical and biological indicators evaluation using support vector machines (SVM). Ecological Indicators 91: 149-154.
12. Huang T, Yang R, Huang W, Huang Y, Qiao X (2018) Detecting sugarcane borer diseases using support vector machine. Information Processing in Agriculture 5(1): 74-82.
13. Ames M, Spooner DM (2008) DNA from herbarium specimens settles a controversy about origins of the European potato. American Journal of Botany 95(2): 252-257.
14. (1986) Coniferous K Starch, sugar crops and breeding. Ankara Univ. Faculty of Agriculture. Ankara, Turkey
15. FAO (2016) Production statistical (potato).
16. Türkiye İstatistik Kurumu (2017) Bitkisel üretim istatistikleri (patates).
17. Yılmaz G (2016) Türkiye’de nişasta - şeker bitkileri üretimi ve sektörün geliştirilmesine dair bazı yaklaşımlar. Vizyon 2023, “2023–2071 Vizyonuyla Tarım” Bildirileri Kitabı, Toçbir-Sen, Ankara, pp. 226-233.
18. Yılmaz G, Kınay A, Kandemir N, Dökülen Ş (2016) Effects of growing conditions on crossing success in different potato crosses. Agro for International Journal 1(3): 133-142.
19. Poehlman JM, Sleeper DA (1995) Breeding field crops. Iowa State University Press. USA, pp. 419.
20. Yılmaz G, Karan Y, Kandemir N, Yanar Y (2013) Orta kuzey geçit bölgesine uygun patates (Solanum tuberosum l.) ıslah çalışmalarında stabilite analizi. 10. Tarla Bitkileri Kongresi Sunulu Bildiriler, Eylül, Konya, p. 10- 13 .
21. Struik PC, Wiersema SG (1999) Seed potato technology. Wageningen Pres, pp. 383.
22. Tarn TR, Tai GCC, De Jong H, Murphy AM, Seabrook JEA (1997) Breeding potatoes for long-day temperate climates. J Janick (Eds.). In: Plant Breeding Reviews. John Wiley&Sons, Inc., New York 9: 217-331.
23. Vapnik VN (1995) The nature of statistical learning theory. Information Science and Statistics, Springer, New York.
24. Osowski S, Siwek K, Markiewicz T (2004) MLP and SVM networks-a comparative study. In Signal Processing Symposium, NORSIG 2004. IEEE. Proceedings of the 6th Nordic, p. 37-40.
25. Shen X, Lin Y (2004) Gene expression data classification using SVM-KNN classifier. In Intelligent Multimedia, Video and Speech Processing, IEEE Proceedings of 2004 International Symposium, pp. 149-152.
26. Kavzoğlu T, Çölkesen İ (2010) Destek vektör makineleri ile uydu görüntülerinin sınıflandırılmasında kernel fonksiyonlarının etkilerinin incelenmesi. Harita Dergisi 144(7): 73-82.
27. Polat K, Güneş S (2007) Breast cancer diagnosis using least square support vector machine. Digital Signal Processing 17(4): 694-701.
28. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE transactions on Neural Networks 13(2): 415-425.
29. Kim KJ (2003) Financial time series forecasting using support vector machines. Neurocomputing 55(1-2): 307-319.
30. Nasrabadi NM (2007) Pattern recognition and machine learning. Journal of electronic imaging 16(4): 049901.
31. Vapnik VN (2000) The nature of statistical learning theory, ser. Statistics for engineering and information science. Springer, New York 21(2): 1003-1008.
32. Osuna E, Freund R, Girosi F (1997) An improved training algorithm for support vector machines. In Neural Networks for Signal Processing. 7th Proceedings of the 1997 IEEE Workshop, pp. 276-285.
33. Bellanger M (2000) Digital processing of signal: theory and practice. John Wiley and Sons, USA, pp. 456.
34. Demuth H, Beale M (2000) Neural network toolbox user’s guide.
35. Alkan A (2001) Predictive data mining with neural networks and genetic algorithms, Ph.D. Thesis, İstanbul Technical University. Institute of Science and Technology, Turkey.
36. Kaastra I, Boyd M (1996) Designing a neural network for forecasting financial and economic time series. Neurocomputing 10(3): 215-236.
37. Nabiyev VV (2003) Artificial inteligence problems-methods-algorithms. Seçkin Publishing, Ankara, Turkey.