title: LDA with stepwise feature selection in caret
body: <p>I'm running a linear discriminant analysis on a few hundred variables and am using caret's 'train' function with the built in model 'stepLDA' to select the most 'informative' variables. This is one of several model types I'm building to test.</p>
  
  <p>Basically, when I call the output object (which is of type <em>train</em>) I get an object similar to this (smaller example model):</p>
  
  <pre><code>Linear Discriminant Analysis with Stepwise Feature Selection 
  
  72 samples
  71 predictors
  2 classes: 'Cancer', 'Normal' 
  
  No pre-processing
  Resampling: Cross-Validated (3 fold, repeated 1 times) 
  Summary of sample sizes: 48, 48, 48 
  Resampling results
  
    Accuracy   Kappa      Accuracy SD  Kappa SD 
    0.6388889  0.2777778  0.06364688   0.1272938
  
  Tuning parameter 'maxvar' was held constant at a value of 50
  Tuning parameter 'direction' was held constant at a value of forward
  </code></pre>
  
  <p>However, when I call the final model (which is of class <em>stepclass</em>) I get:</p>
  
  <pre><code>method      : lda
  final model : y ~ Age + X4 + X167 + X184 + X274 + X416 + X548 + X571 + X594
  + X611 + X618 + X634 + X1094 + X1632 + X1921 + X2437 + X3018 + X3474
  &lt;environment: 0x000000001dad6890&gt;
  
  accuracy = 0.8538
  </code></pre>
  
  <p>My question is this: to what does the accuracy of 0.6388889 refer. I know that it is a cross-validated accuracy, but is it from LDA models built on all 71 predictors first before stepwise selection is performed? Or is there another step going on in there?</p>
  
  <p>I know that the 0.8538 refers to the 10-fold cross-validated accuracy of the final selected model on the training dataset. I'm trying to figure out which is the appropriate accuracy to report for the model??</p>
  
  <p>Thanks for any help anyone can give :)</p>