title: Gaussian RBF Visualization in Python
body: <p>It is some days that I am trying to visualize the so-called kernel trick resulting from a RBF kernel transformation in a SVC model. Basically, I am trying to map a 2D space to a 3D space in order to let the viewer see how the kernel trick adds a dimension in order to linearly separate the space between two classes.</p>
  <p>Following sklearn examples, I managed to plot a 2D example of the trick. However, I feel it is not enough to really grasp what it is happening behind the scences.</p>
  <p>Below is what I managed to plot:</p>
  <p><a href="https://i.stack.imgur.com/W0OVx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/W0OVx.png" alt="enter image description here" /></a></p>
  <p>I would like to plot the same data on a three dimensional space, representing also the plane that splits the space between the two classes.
  I am not asking for the actual code here. Rather, I would like to understand what goes on the axis for the third dimension. Indeed, I think that such axis should be equal to <code>exp(-gamma||x-y||^2)</code>. However, due to my poor vector algebra skills, I do not know how to compute it.</p>
  <p>Any help would be much appreciated.</p>
  <p>Cheers!</p>
  <p><em><strong>UPDATE</strong></em></p>
  <p>The following allowed me to build a new matrix for a 3D plot:</p>
  <pre><code>def feature_map_2(X):  
      return np.asarray((X[:,0], X[:,1], np.exp( -gam*(( X[:,0]**2 + X[:,1]**2 -2*X[:,0]*X[:,1]))))).T
  
  Z = feature_map_2(X)
  </code></pre>
  <p>Where <code>gam = 1/n_features</code></p>
  <p>Then, I computed the boundary as follows:</p>
  <pre><code>#SVM 
  clf = svm.NuSVC(kernel = 'linear', nu=0.5)
  clf.fit(Z, y) 
  
  
  w = clf.coef_.flatten()
  b = clf.intercept_.flatten()
  
  # create x,y
  xx, yy = np.meshgrid(np.linspace(-6,6), np.linspace(-2,2))
  
  # calculate corresponding z
  boundary = lambda xx, yy: (-w[0] * xx - w[1] * yy - b) * 1. /w[2]
  </code></pre>
  <p>However, results differ from what one might have expected looking at the 2D plot.</p>