title: How to change the performance metric from accuracy to precision, recall and other metrics in the code below?
body: <p>As a beginner in scikit-learn, and trying to classify the iris dataset, I'm having <em>problems with adjusting the scoring metric</em> from <code>scoring='accuracy'</code> to <em>others like precision, recall, f1</em> etc., in the cross-validation step. Below is the <strong>full</strong> code sample (<strong>enough to start at</strong> <code># Test options and evaluation metric</code>).</p>
  
  
  
  <pre class="lang-python prettyprint-override"><code># Load libraries
  import pandas
  from pandas.plotting import scatter_matrix
  import matplotlib.pyplot as plt
  from sklearn import model_selection # for command model_selection.cross_val_score
  from sklearn.metrics import classification_report
  from sklearn.metrics import confusion_matrix
  from sklearn.metrics import accuracy_score
  from sklearn.linear_model import LogisticRegression
  from sklearn.tree import DecisionTreeClassifier
  from sklearn.neighbors import KNeighborsClassifier
  from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
  from sklearn.naive_bayes import GaussianNB
  from sklearn.svm import SVC
  
  
  
  # Load dataset
  url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/iris.csv"
  names = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width', 'class']
  dataset = pandas.read_csv(url, names=names)
  
  
  # Split-out validation dataset
  array = dataset.values
  X = array[:,0:4]
  Y = array[:,4]
  validation_size = 0.20
  seed = 7
  X_train, X_validation, Y_train, Y_validation = model_selection.train_test_split(X, Y, test_size=validation_size, random_state=seed)
  
  
  # Test options and evaluation metric
  seed = 7
  scoring = 'accuracy'
  
  
  #Below, we build and evaluate 6 different models
  # Spot Check Algorithms
  models = []
  models.append(('LR', LogisticRegression()))
  models.append(('LDA', LinearDiscriminantAnalysis()))
  models.append(('KNN', KNeighborsClassifier()))
  models.append(('CART', DecisionTreeClassifier()))
  models.append(('NB', GaussianNB()))
  models.append(('SVM', SVC()))
  
  
  # evaluate each model in turn, we calculate the cv-scores, ther mean and std for each model
  # 
  results = []
  names = []
  for name, model in models:
      #below, we do k-fold cross-validation
      kfold = model_selection.KFold(n_splits=10, random_state=seed)
      cv_results = model_selection.cross_val_score(model, X_train, Y_train, cv=kfold, scoring=scoring)
      results.append(cv_results)
      names.append(name)
      msg = "%s: %f (%f)" % (name, cv_results.mean(), cv_results.std())
      print(msg)
  </code></pre>
  
  <p>Now, apart from scoring ='accuracy', I'd like to evaluate other performance metrics for this multiclass classification problem. But when I use, scoring='precision', it raises:</p>
  
  <pre class="lang-python prettyprint-override"><code>ValueError: Target is multiclass but average='binary'. Please choose another average setting.
  </code></pre>
  
  <p><strong>My questions are:</strong></p>
  
  <p>1) I guess the above is happening because 'precision' and 'recall' are defined in scikit-learn only for binary classification-is that correct? If yes, then, which command(s) should replace <code>scoring='accuracy'</code> in the code above?</p>
  
  <p>2) If I want to compute the confusion matrix, precision and recall for each fold while performing the k-fold cross validation, what commands should I type? </p>
  
  <p>3) For the sake of experimentation, I tried scoring='balanced_accuracy', only to find:</p>
  
  <pre class="lang-python prettyprint-override"><code>ValueError: 'balanced_accuracy' is not a valid scoring value.
  </code></pre>
  
  <p><em>Why is this happening, when the model evaluation documentation (<a href="https://scikit-learn.org/stable/modules/model_evaluation.html" rel="nofollow noreferrer">https://scikit-learn.org/stable/modules/model_evaluation.html</a>) clearly says balanced_accuracy is a scoring method</em>? I'm quite confused here, so an actual code to show how to evaluate other performance etrics would be appreciated! Thanks inn advance!!</p>