title: Choose a combination of variables using cross validation and stepwise regression
body: <p>I have a dataset, called cost, that have roughly 400 observations, the dependent variable Y and ten independent variables. Two of them, PB1 and PB2, have to be in the model. Among the remaining eight variables, I want to choose one among V0, V1, V2 and V3 and one among L0, L1, L2 and L3 that better fit the model. Meanwhile, I also want to cross validate the model. I tried to use the forward stepwise regression and CV, but it seems the model may include V0 and V1 (or L1 and L3, etc) in the same model, which does not fit my purpose. Any suggestion on how to code it in R? </p>
  
  <p>Below is my code so far:</p>
  
  <pre><code>set.seed(6)
  folds = sample(rep(1:10, length = nrow(cost)))
  cv.errors = matrix(NA, 5, 10)
  for (k in 1:5){
    best.fit = regsubsets(Y~., data = cost[folds!=k,], nvmax = 10, method = "forward")
    for(i in 1:5){
      pred = predict(best.fit, cost[folds == k,], id = i)
      cv.errors[k, i] = mean((cost$Y[folds ==k] - pred)^2)
    }
  }
  rmse.cv = sqrt(apply(cv.errors, 2, mean))
  plot(rmse.cv, pch = 19, type = "b")
  </code></pre>
  
  <p>There is no predict function in regsubsets. Instead, predict.regsubsets was defined as a function prior such as</p>
  
  <pre><code>predict.regsubsets = function(object, newdata, id, ...) {
    form = as.formula(object$call[[2]])
    mat = model.matrix(form, newdata)
    coefi = coef(object, id = id)
    mat[, names(coefi)] %*% coefi
  }
  </code></pre>