title: Controlling sampling for crossvalidation in the caret R package
body: <p>I have the following problem. In a data set from N subjects I have several samples per subject. I want to train a model on the data set, but I would like to make sure that in each resampling, in the training set there are no replicates of the subjects.</p>
  
  <p>Alternatively, I would block the cross-validation by subject. Is that possible?</p>
  
  <p>Without the caret package, I would do something like that (mock code)</p>
  
  <pre><code>subjects &lt;- paste0("X", 1:10)
  samples  &lt;- rep(subjects, each=5)
  x &lt;- matrix(runif(50 * 10), nrow=50)
  loocv &lt;- function(x, samples) {
    for(i in 1:nrow(x)) {
       test &lt;- x[i,]
       train &lt;- x[ samples != samples[i],]
       # create the model from train and predict for test
    }
  }
  </code></pre>
  
  <p>or, alternatively, </p>
  
  <pre><code>looSubjCV &lt;- function(x, samples, subjects) {
     for(i in 1:length(subjects)) {
       test &lt;- x[ samples == subjects[i], ]
       train &lt;- x[ samples != subjects[i], ]
       # create the model from train and predict for test
    }
  }
  </code></pre>
  
  <p>Otherwise, the presence of other samples from the same subject will result in overfitting of the model.</p>