title: taking the gradient in Tensorflow, tf.gradient
body: <p>I am using this function of tensorflow to get my function jacobian. Came across two problems:</p>
  
  <ol>
  <li>The tensorflow documentation is contradicted to itself in the following two paragraph if I am not mistaken:</li>
  </ol>
  
  <blockquote>
    <p>gradients() adds ops to the graph to output the partial derivatives of ys with respect to xs. It returns a list of Tensor of length len(xs) where each tensor is the sum(dy/dx) for y in ys.
    Blockquote</p>
    
    <p>Blockquote
    Returns:
    A list of sum(dy/dx) for each x in xs.
    Blockquote</p>
  </blockquote>
  
  <p>According to my test, it is, in fact, return a vector of len(ys) which is the sum(dy/dx) for each x in xs. </p>
  
  <ol start="2">
  <li><p>I do not understand why they designed it in a way that the return is the sum of the columns(or row, depending on how you define your Jacobian). </p></li>
  <li><p>How can I really get the Jacobian? </p></li>
  </ol>
  
  <p>4.In the loss, I need the partial derivative of my function with respect to input (x), but when I am optimizing with respect to the network weights, I define x as a placeholder whose value is fed later, and weights are variable, in this case, can I still define the symbolic derivative of function with respect to input (x)? and put it in the loss? ( which later when we optimize with respect to weights will bring second order derivative of the function.)</p>