title: K-fold cross-validation for testing model accuracy in MATLAB
body: <p>I'm having some trouble truly understanding what's going in MATLAB's built-in functions of cross-validation. My goal is to develop a model for binary classification and test its accuracy by using cross-validation. I'm looking at comparing a few different models, but I'll just use k-nearest neighbor classification for the purposes of this example. I'm providing code for two simplified versions of cross-validation: </p>
  
  <pre><code>% INPUTS:
  % data = NxM matrix, where N = observations and M = features
  % labels = Nx1 matrix with the true labels of each observation
  
  %% Version 1
  model = fitcknn(data,labels,'NumNeighbors',kVal); %kVal is some iteratively-determined value
  
  % Calculate misclassification error
  cvmodel = crossval(model,'kfold',10);
  cvError = kfoldLoss(cvmodel);
  
  % Generate confusion matrix    
  predictedLabels = predict(model,data);
  R = confusionmat(predictedLabels,labels);
  
  
  %% Version 2
  indices = crossvalind('kfold',labels,10);
  confusionMatrix = cell(1,1);
  errorMat = zeros(1,10);
  for i = 1:10
      test = (indices==i);
      train = ~test;
      knn = fitcknn(data(train,:),labels(train),'NumNeighbors',kVal);
      y = knn.predict(data(test,:));
      index = cellfun(@strcmp,y,labels(test));
      errorMat(i) = sum(index)/length(y);
      confusionMatrix{i} = confusionmat(labels(test),y);
  end
  
  % Calculate misclassification error
  cvError = 1-mean(errorMat); 
  
  % Generate confusion matrix
  R = cat(3,confusionMatrix{:});
  R = sum(R,3);
  </code></pre>
  
  <p>Which of these two approaches is the correct implementation? Or are both equivalent? This question can be generalized to other types of classification models in MATLAB, e.g. discriminant analysis. Appreciate any and all input, thanks in advance!</p>