title: SVM: Cost parameter VS. number of support vectors
body: <p>I am using the library e1071 to train SVM model in R, where i change the cost function and observe the number of resulting Support vectors.</p>
  
  <pre><code>library("e1071")
  library("mlbench")
  data(Glass, package="mlbench")
  svm.model &lt;- svm(Type ~ ., data = Glass, cost = 0.00100, gamma= 1)
  sum(svm.model$nSV)
  #[1] 208
  svm.model &lt;- svm(Type ~ ., data = Glass, cost = 1, gamma= 1)
  sum(svm.model$nSV)
  #[1] 183
  svm.model &lt;- svm(Type ~ ., data = Glass, cost = 100000, gamma= 1)
  sum(svm.model$nSV)
  #[1] 172
  </code></pre>
  
  <p>My question is the following:
  Is the cost parameters here equivalent to the C parameter in the dual Lagrange formulation of the soft margin SVM?
  If those parameters are the same, then should not we observe an increasing number of support vectors?</p>
  
  <p>"However, it is critical here, as in any regularization scheme, that a proper value is chosen for C, the penalty factor. If it is too large, we have a high penalty for nonseparable points and we may store many support vectors and overfit. If it is too small, we may have underfitting."
  Alpaydin (2004), page 224</p>
  
  <p>The presented example shows that the greater the cost parameter is the less support vectors we get. So what is wrong here?</p>
  
  <p>[EDIT 1] I Exchange some emails with the editors of the mentioned library and he gave a counterexample.</p>
  
  <p>"Basically, yes, but this is not linear, try: "</p>
  
  <p>N = sapply(1:1000, function(i) svm(Species ~ ., data = iris, cost = i)$tot.nSV)
  plot(N) </p>