title: multi-variable Gradient descent using Sympy
body: <p>I'm practicing on Gradient descent algorithm implementation for two variables in Sympy library in Python 2.7.   </p>
  
  <p>My goal is to find minimum of two variable function using vector of derivatives according to following steps:</p>
  
  <ol>
  <li>For function f(a,b) of two varibale define the Matrix of first
  partial differentials - <em>M</em>. </li>
  <li>Then, I pass the the statring point of a,b (for instance <em>V0</em> = (1.0,1.0)) into M and multiply this by step - this gives  <em>M0</em> *  matrix. </li>
  <li>Next, substract the result calculated above from vector of
  starting values - <em>V0</em>. This gives a new vector of variables a,b - <em>V1</em>. </li>
  <li>Finally, values of <em>V1</em> are put in M again. If results of matrix <em>M</em> are  less than epsilon = > continue the iterations.</li>
  </ol>
  
  <p>The code and steps description is attached. I suppose the problem lies in while loop, since it gives the worng values and algorithm takes the first i iteration only. 
  <a href="https://i.stack.imgur.com/rCLUt.png" rel="nofollow noreferrer">enter image description here</a></p>
  
  <p>Could you please advise on improvements? </p>
  
  <pre><code>import sympy as sp
  from sympy import *
  from sympy import lambdify
  
  a=Symbol('a')
  b=Symbol('b')
  alpha=Symbol('alpha')
  
  def test_f(a, b):
      # define a test function of a,b
      test_f=4*a**2 + 25*b**2 
      return test_f
  
  def deriv_matrix(f, a, b):
      # define the matrix of derivatives
      d1=diff(f(a,b), a, 1)
      d2=diff(f(a,b), b, 1)
      M=Matrix([d1,d2])
      return M
  
  epsilon=0.02
  alpha=0.1
  i=1 # strat the iteration from 1
  vector1=Matrix([1,1]) # starting point 
  
  while (i&lt;100) &amp; (f(vector1[0], vector1[1]).all()&gt; epsilon):
      f = lambdify((a,b), deriv_matrix(test_f, a, b))
      vector=vector1
      N=-(alpha/i)*f(vector[0],vector[1])
      vector1=vector1+N
      i+=i
  print vector1
  </code></pre>