title: Compute the gradient of the SVM loss function
body: <p>I am trying to implement the SVM loss function and its gradient.
  I found some example projects that implement these two, but I could not figure out how they can use the loss function when computing the gradient.</p>
  
  <p>Here is the formula of loss function:
  <a href="https://i.stack.imgur.com/qGd90.png" rel="noreferrer"><img src="https://i.stack.imgur.com/qGd90.png" alt="enter image description here"></a></p>
  
  <p>What I cannot understand is that how can I use the loss function's result while computing gradient?</p>
  
  <p>The example project computes the gradient as follows:</p>
  
  <pre><code>for i in xrange(num_train):
      scores = X[i].dot(W)
      correct_class_score = scores[y[i]]
      for j in xrange(num_classes):
        if j == y[i]:
          continue
        margin = scores[j] - correct_class_score + 1 # note delta = 1
        if margin &gt; 0:
          loss += margin
          dW[:,j] += X[i]
          dW[:,y[i]] -= X[i] 
  </code></pre>
  
  <p>dW is for gradient result. And X is the array of training data.
  But I didn't understand how the derivative of the loss function results in this code. </p>