Comparative Study of Data Mining Classifiers with Different Features and Different Databases Domain

P. Arumugam Poompavai A, Manimannan G, Poongothai Associate Professor, Department of Statistics, Annamalai University, Chidhambaram, Assistant Professor, Department of Statistics, Apollo Arts and Science College, Chennai. Assistant Professor, Department of Mathematics, TMG College of Arts and Sciences, Chennai. Assistant Professor, Department of Statistics, Apollo Arts and Science College, Chennai

Similarly, genetic algorithms and neural networks areused for optimisation of empirically observed functions under constraints. Data mining is an iterative process in all fields to discover Knowledge Discovery Database (KDD).
Statisticians mostly analyzed systemically planned experiments to reply to a thoroughly formulated scientific question. These experiments lead to small amount of high quality data. Under these controlled conditions one could often derive an optimal way of collecting and analyzing the data and mathematically prove this property. The scale of data set has changed. Data are growing in two dimensions: they not only consist of more and more observations, they also contain more and more variables. Often these data are not directly sampled (for analysis)., but are merely by product of other activities. As such, they do not necessarily stem from good experimental design and some variable might contain no information. The data thus contains more and more 'noise'.

Figure1. KDD of Data Mining Processes
Thus data mining differs from traditional statistics in several ways: formal statistical inference is assumption driven in the sense that a hypothesis is formed and validated against the data (Figure 1). Data mining in contrast is discovery driven in the sense that patterns and hypothesis are automatically extracted from data, another way, data mining is data driven, while statistics is human driven. The branch of statistics that data mining resembles most is exploratory data analysis, although this field, like most of the rest of statistics, has been focused on data sets far smaller than most that are the target of data mining researchers. Data mining also differs from traditional statistics in that sometimes the goal is to extract qualitative models which can easily be translated into logical rules or visual representations; in this sense data mining is human centered and is sometimes coupled with humancomputer interfaces research (J. Han, M, Kamber and J Pei, 2012).

REVIEW OF LITERATURE
In recent days the amount of data stored in educational database is increasing rapidly. Brijesh Kumar Bhardwaj, Saurabh Pal used Bayes classification prediction model to identify the difference between high learners and slow learners student. U. Rajendra Acharya, P. Subbanna Bhat, S.S. Iyengar , Ashok Rao, Sumeet Dua dealt with the classification of certain diseases using artificial neural network (ANN) and fuzzy equivalence relations. The heart rate variability is used as the base signal from which certain parameters are extracted and presented to the ANN for classification. The same data is also used for fuzzy equivalence classifier. The feedforward architecture ANN classifier is seen to be correct in about 85% of the test cases, and the fuzzy classifier yields correct classification in over 90% of the cases.
The successful application of data mining in highly visible fields like e-business, marketing and retail has led to its application in other industries and sectors. Among these sectors just discovering is healthcare. The healthcare environment is still "information rich" but "knowledge poor". Wealth of data is available within the healthcare systems. However, there is a lack of effective analysis tools to discover hidden relationships and trends in data. This research paper intends to provide a survey of current techniques of knowledge discovery in databases using data mining techniques that are in use in today's medical research particularly in Heart Disease Prediction. Number of experiment has been conducted to compare the performance of predictive data mining technique on the same dataset and the outcome reveals that Decision Tree outperforms and some time Bayesian classification is having Comparative Study of Data Mining Classifiers with Different Features and Different Databases Domain similar accuracy as of decision tree but other predictive methods like KNN, Neural Networks, Classification based on clustering are not performing well. The second conclusion is that the accuracy of the Decision Tree and Bayesian Classification further improves after applying genetic algorithm to reduce the actual data size to get the optimal subset of attribute sufficient for heart disease prediction. (Jyoti Soni, Ujma Ansari, Dipesh Sharma, Sunita Soni) Text classification is the process of classifying documents into predefined categories based on their content. It is an automated assignment of natural language texts to predefined categories. Text classification is primary requirement of text retrieval systems, which retrieve texts in response to a user query, and text understanding systems, which transform text in some way such as producing summaries, answering questions or extracting data. Existing supervised learning algorithms to automatically classify text need sufficient documents to learn accurately. A new algorithm for text classification using data mining that requires fewer documents for training. Instead of using words and word relation association rules from these words is used to derive feature set from pre-classified text documents. The concept of Naïve Bayes classifier is then used on derived features and finally only a single concept of Genetic Algorithm has been added for final classification. A system based on the proposed algorithm has been implemented and tested. The experimental results show that proposed system works as a successful text classifier. (S. M. Kamruzzaman, Farhana Haider, Ahmed Ryadh Hasan) Another research paper describes about the performance analysis of different data mining classifiers before and after feature selection on binomial data set. Three data mining classifiers Logistic Regression, SVM and Neural Network classifiers are considered for classification. The Congressional Voting Records data set is a binomial data set investigated in this study is taken from UCI machine learning repository. The classification performance of all classifiers is presented by using statistical performance measures like accuracy, specificity and sensitivity. Gain chart and R.O.C (Receiver Operating Characteristics) chart are also used to measure the performances of the classifiers. A comparative study is carried out among the data mining classifiers. Experimental result showed that without feature selection Logistic Regression and SVM classifiers provides 100% accuracy and neural network provides 98.13 % accuracy on test data set. With feature selection SVM classifier provides 100% accuracy. The performance of SVM classifier is found to be the best among all the classifiers with reduced number of features. (Pushpalata Pujari) 4. DATABASES

Dataset 1
The secondary database was collected from UCI website. The number of instances in this study is 650 and number of attributes are 32. The attributes used in this study are school, internet, romantic, address, gender, age, parent status, mother education, father education, travel time to school from home, study time, activities, health and absences. These data mining classification model were developed using R language. Initially dataset had 32 attributes. After attribute selection (internet, romantic, address, sex, age, Status, Medu, Fedu, traveltime, studytime, activities, health and absences) missing values records were identified and were deleted from dataset. After deleting records with missing values, 649 were left out. On these 649 records data mining classification such as techniques, Random Forest, Support Vector Machine (SVM) and Naive Bayes Analysis were applied.

Dataset 2
The data is a secondary data and taken from DATA.GOV website. The number of instances in this study is 1565 and number of attributes are 13. The attributes used in this study are state, record test iodine, age, bmi, hb, fasting sugar. The data mining classification model were developed using R language. Initially dataset had 13 attributes. After attribute selection (state, area, age, record test iodine, bmi, hb, fasting sugar) missing values records were identified and were deleted from dataset. After deleting records with missing values we were left with 1565 records. On these 1565 records data mining classification techniques such as Random Forest, Support Vector Machine (SVM) and Naive Bayes Analysis were applied.

Dataset 3
The data is a secondary data and taken from UCI website. The number of instances in this study is 4521 and number of attributes are 17. The attributes used in this study are age, job, marital status, education, default, housing, loan, contact, day, month, duration, campaign, poutcome and dependent variable. The data mining classification model were developed using R language. Initially dataset had 17 attributes. After attribute selection (age, job, marital status, education, default, balance, housing, loan, contact, day, month, duration, campaign, pdays, previous, poutcome and dependent variable) missing values records were identified and were deleted from dataset. After deleting records with missing values we were left with 4522 records. On these 4522 records data mining classification techniques such as Random Forest, Support Vector Machine (SVM) and Naive Bayes Analysis were applied.
In this study, three datasets were considered which are from the UCI Repository and DATA.GOV. These datasets are effective enough to show classification process. These datasets are analysed under different classification parameters. The detailed description of these databases is given below ( Every dataset has different types of data, including numbers, text and other domain data points. Each of the dataset is explored explicitly due to their uniqueness in terms of their varying attributes, discrete or continuous nature of data etc. These datasets are analyzed for classification task by using R programming tool under different classification approaches.
R programming contains number of built-in data mining classification so that different mining operations can be performed directly. R is used by researches to analyze effectiveness of different algorithms. In this study, R tool is used to perform analytical study of classification on datasets.

Random Forest
The random forests algorithm is a machine learning technique that is increasingly being used for image classification and creation of continuous variables such as percent tree cover International Conference on Geo-informatics for Spatial Infrastructure Development in Earth and Allied Sciences 2010 and forest biomass. Random forests are an ensemble model which means that it uses the results from many different models to calculate a response. In most cases the result from an ensemble model will be better than the result from any one of the individual models (Dahinden 2009). In case of random forests, several decision trees are created (grown) and response is calculated based on the outcome of all the decision trees ( Figure 2).

Naive Bayes
Naive Bayes is a classification technique based on Bayes' Theorem with an assumption of independence among predictors. In simple terms, a Naive Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature. Naive Bayes model is easy to build and particularly useful for very large data sets. Along with simplicity, Naive Bayes is known to outperform even highly sophisticated classification methods (Figure 3).

Naive Bayes Algorithm
Bayes theorem provides a way of calculating posterior probability P(c|x) from P(c), P(x) and P(x|c). The equation is: Where, P(c|x) is the posterior probability of class (c, target) given predictor (x, attributes). P(c) is the prior probability of class. P(x|c) is the likelihood which is the probability of predictor given class.
P(x) is the prior probability of predictor.

Support Vector Machine
A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyper plane. In other words, given labelled training data (supervised learning), the algorithm outputs an optimal hyper plane which categorizes new examples. There are many classifiers that originated in statistics. Examples, naive Bayes classifier, maximum entropy classifier, Fisher's discriminant classifier, partial least squares classifier, and Mahalanobis distance based classifier. In addition, multiple (linear and nonlinear) regression and logistic regression models can be used as classifier. Some of these classical models for pattern classification and prediction have assumptions on the data distributions.
For instances, multiple regression models assume that error terms are normally distributed, and that independent variables are correlated. Similarly, normality is assumed in discriminant analysis, canonical correlation, etc. The Support Vector Machine (SVM) is a supervised classification model without any assumptions on the data distribution. Another name for SVM is kernel machines (as nonlinear SVM uses a kernel mapping). A machine learning algorithm tries to learn the relationship (X→y) from the training data X to the classes or categories y, so that it can be used to classify new data instances. It is used for pattern recognition (eg: face, retina, fingerprint and other images, handwritings and speech recognition), classification (eg: medical classification), clustering (web page and image clustering) and regression (SVR). There could exist multiple separating hyper plane when the number of data points is larger than the dimensionality (Figure 4). In this dataset, the researcher compared three data mining classifier based on their sensitivity, specificity and accuracy. It shows that SVM classifier has better classification precision compared with other classifier ( Figure 5, Table 2). In this dataset, three data mining classifier based on their sensitivity, specificity and accuracy was compared and found that SVM classifier has better classification precision than other classifier (Table 4, Figure 6). The results shows that SVM outperforms well than Decision Tree, Random Forest, Naive Bayes, LDA models, parameters Sensitivity, Specificity, Accuracy and Error Rates/.

Dataset 3
In .In classification tree, the variables used in tree construction for the data are day, duration, job, marital status, month, pdays and poutcome. The root node error is 0.11524. Data based on classification when considering decision tree, the tree construction represents the duration, with poutcome and marital status. On further classification, it is been grouped based on month, pdays, and job. In this dataset, three data mining classifier based on their sensitivity, specificity and accuracy were compared. The experiment proved that SVM classifier has better classification precision than other classifiers. The results shows that out of Decision Tree, Random Forest, Naive Bayes, SVM and LDA models, parameters Sensitivity, Specificity, Accuracy and Error Rates, SVM outperforms well.

Figure7. Graphical Representations of Sensitivity, Specificity, and Accuracy
The results shows that out of Random Forest, Naive Bayes and SVM models, parameters Sensitivity, Specificity, Accuracy and Error Rates SVM outperforms well. A distinguished confusion matrix was obtained to calculate sensitivity, specificity and accuracy. Confusion matrix is a matrix representation of the classification results. The table below shows the confusion matrix. (Table 8)

Sensitivity, Specificity and Accuracy Calculation
Below formulae were used to calculate sensitivity, specificity and accuracy: Performance analysis was carried out on five different data mining classifier for three different datasets. Datasets considered are from survey domain. The present work has been implemented in R programming language environment and the results have been taken under different parameters: the sensitivity, accuracy and Kappa Statistic. The results obtained from these different models have been defined in the form of tables as well as graph.  Figure 7 for Comparison of Sensitivity, Specificity, and Accuracy for four Databases It is clear that figure 7 shows the accuracy based comparison of different classification. It shows that SVM is most robust, effective, and consistent classifier for different datasets. SVM provides higher accuracy among all classification whereas Naive Bayes is the least effective classification in terms of accuracy analysis.

Figure9. Comparisons Charts for four Databases using Kappa Statistics
Kappa Statistics is a statistical analysis based on inter-ratter agreement for qualitative data. It basically performs the analysis between different classes. Higher Value of kappa statistic is considered as good (Table 11). Figure 9 shows the comparative analysis of different classification under the kappa statistics.

CONCLUSION
This paper focuses on various classification techniques used in data mining and a study on each of them. Data mining can be used in a wide area that integrates techniques from various fields including machine learning, Network intrusion detection, spam filtering, artificial intelligence, statistics and pattern recognition for analysis of large volumes of data. Classification methods are typically strong in modeling communications. Classification is the preliminary stage of data mining which is used to categorize dataset in smaller groups where each group contains similar data items. The classification basically deals with two main parameters in which one is the number of classes and another is the criteria for deciding the class members. The accuracy of classification algorithm also decides the effectiveness of its use in other mining applications. The present work is about to analyze the effectiveness of most popular classification techniques. In this paper, analysis has been performed for three different classification methods in terms of precision, accuracy, and kappa statistics under three datasets, collected from different domain. The work has been implemented in R programming language environment and obtained results show that SVM is the most robust classification method. Due to the nature of some data sets, the result reveals that all data mining techniques accomplish their goals perfectly, but each technique has its own characteristics and specification that demonstrate their precision, accuracy, proficiency and preference. In Addition, Support Vector Machines, Naïve Baye