anchor.diff:  stem:[L(w_1,w_2) = (w_1+2)^2+4(w_2-1)^2+3]
   
   To perform gradient descent, we need to compute the gradient of this loss function.
  -The gradient is just the set of partial derative of the loss function for the weight parameters.
  +The gradient is just the set of partial derivative of the loss function for the weight parameters.
   
   stem:[
   \nabla L(w_1,w_2) =
  
positive.code: 
  It’s not a straight forward job. The problem is that the loss function is on top of the network which is a giant non linear function. We can’t compute the minimum right away from the data, network and loss function.
  
  Ok, how do we do then ? The answer is Gradient descent.
  
  == Gradient descent
  
  
negative.code: and stem:[B \in \mathbb{R}]
  
  This function simply computes a weighted sum of the input data,
  adds a bias and saturates the result with stem:[\max()]. The stem:[\max()] part
  is called the activation function.
  There are different possible activation functions.
  
  
  