title: Hyperparameter tuning one-class SVM
body: <p>I am looking for a package or a 'best practice' approach to automated hyper-parameter selection for one-class SVM using Gaussian(RBF) kernel. I am currently implementing libsvm's one-class svm in R so preferably an approach incorporating that or, at least, R would be best.</p>
  <hr />
  <p>EDIT</p>
  <p>Just to give a more clear example of what I am looking for, let's say we have the iris dataset and we take one of the types as the positive cases. One approach is to build a one-class SVM with differing choices of nu and gamma and then to validate the accuracy of the model against the negative cases (the other flower types). See below:</p>
  <pre><code>library(datasets)
  library(data.table)
  library(e1071)
  #load the iris data
  data(iris)
  #separate positive and negative cases
  positive_cases &lt;- iris[iris$Species==&quot;virginica&quot;,1:4]
  negative_cases &lt;- iris[iris$Species!=&quot;virginica&quot;,1:4]
  #get hyperparameter choices
  hyp_param_choices &lt;- setDT(expand.grid(&quot;nu&quot;=seq(.1,.3,by=.1),
                                         &quot;gamma&quot;=1*10^seq(-2, 2, by=1)))
  hyp_param_choices[,err:=0]
  
  for(hyp_i in 1L:nrow(hyp_param_choices)){
    tuned &lt;- svm(x=positive_cases, 
                 y=rep(T,nrow(positive_cases)), #True as they are all in the positive class
                 nu =  hyp_param_choices[hyp_i,nu],
                 gamma = hyp_param_choices[hyp_i,gamma],
                 type='one-classification',
                 scale=T #scale the data
    )
    svm_neg_pred &lt;- predict(tuned, #predict the negative classes, should all be false
                            negative_cases)
    #error is sum of svm_neg_pred as this counts all the positives .i.e false positive cases divided by total number of negatives
    set(hyp_param_choices, i=hyp_i, j=&quot;err&quot;, value=(sum(svm_neg_pred)/nrow(negative_cases)))
  }
  </code></pre>
  <pre><code>setorder(hyp_param_choices,err)
  print(hyp_param_choices)
       nu gamma  err
   1: 0.1 1e+00 0.00
   2: 0.2 1e+00 0.00
   3: 0.3 1e+00 0.00
   4: 0.1 1e+01 0.00
   5: 0.2 1e+01 0.00
   6: 0.3 1e+01 0.00
   7: 0.1 1e+02 0.00
   8: 0.2 1e+02 0.00
   9: 0.3 1e+02 0.00
  10: 0.3 1e-02 0.01
  11: 0.2 1e-01 0.01
  12: 0.2 1e-02 0.02
  13: 0.3 1e-01 0.02
  14: 0.1 1e-01 0.03
  15: 0.1 1e-02 0.05
  </code></pre>
  <p>Now in truth, my problem has some false positives in the training data. We could incorporate that in the example about by adding a sample of the negatives into the positives and excluding these negatives from the validation testing and then rerunning:</p>
  <pre><code>positive_cases &lt;- rbind(iris[iris$Species==&quot;virginica&quot;,1:4],
                          iris[iris$Species!=&quot;virginica&quot;,1:4][sample(nrow(iris[iris$Species!=&quot;virginica&quot;,]), 
                                                                     10),])
  </code></pre>
  <p>I am looking for another approach to choosing the best one-class hyperparameters in a paper or otherwise that has some reasoning for being a good approach.</p>
  <hr />
  <p>To give some background, I am aware of the original implementation of one-class SVMs by <a href="https://dl.acm.org/doi/10.1162/089976601750264965" rel="nofollow noreferrer">Scholkopf et al.</a> and understand the aim of the approach is to map the one-class data to the feature space corresponding to the kernel and to separate them from the origin with the maximum margin using the hyperplane. In this sense the origin can be thought of as all other classes. I am also aware of SVDD's introduced by <a href="https://link.springer.com/article/10.1023/B:MACH.0000008084.60811.49" rel="nofollow noreferrer">Tax &amp; Duin</a>. The aim here is create the smallest possible data enclosing sphere. Through this approach all points outside the sphere are other classes/outliers. I also know that these two approaches deduce to equivalent minimisation functions when using a Gaussian kernel. Both these approaches use soft-margins allowing for misclassified cases in the one-class too. As they are equivalent I will only talk about OC-SVMs but approaches using SVDDs as an answer would also be greatly appreciated!</p>
  <p>So in my problem, my one-class are positive cases for which I want to optimise nu relating to the proportion of misclassified cases (false positives) and gamma, the width of the gaussian kernel. I n my problem, I know there will be false positives, it is the nature of the problem and can not be detected. I also want to apply multiple OC-SVMs on different datasets so I need an automated approach to tuning nu and gamma based on the proportion of outliers present in the dataset in question and the data's latent features.</p>
  <p>As this problem is essentially unsupervised, I obviously can't use CV in the normal manner with a range of nu and gamma as then the solution with the minimum distance from the origin would be chosen. Just to note I do have negative cases but would rather hold them back if possible from a validation step as if not why bother with a one-class approach at all and why not use a normal two-class classification approach?</p>
  <p>My question is whether anyone has found a package or approach to do this in R? I know there are plenty of approaches in scientific literature including to very promising approaches: <a href="http://Two%20methods%20of%20selecting%20Gaussian%20kernel%20parameters%20for%20one-class%20SVM%20and%20their%20application%20to%20fault%20detection" rel="nofollow noreferrer">DTL</a> and <a href="https://www.it-designers-gruppe.de/presse-detailansicht/autonomously-determining-the-parameters-for-svdd-with-rbf-kernel-from-a-one-class-training-set/216b5e56649ea4ac7676403ffdf62aa9/" rel="nofollow noreferrer">here</a> but these don't seem to have code available barring pseudocode and how to translate this to R and incorporate it with libsvm, for example, seems a big step for my current abilities.</p>
  <p>Any help or suggestions at all would be greatly appreciated!</p>