title: Custom gradients for Flux rather than using Zygote A.D
body: <p>I have a machine learning model where the gradients for the model parameters are analytic and there is no need for automatic differentiation. However, I still want to be able to take advantage of different optimizers in Flux without having to rely on Zygote for the differentiation. Here is some snippets of my code.</p>
  
  <pre><code>W = rand(Nh, N)
  U = rand(N, Nh)
  b = rand(N)
  c = rand(Nh)
  
  θ = Flux.Params([b, c, U, W])
  
  opt = ADAM(0.01)
  </code></pre>
  
  <p>I then have a function that computes analytic gradients of my model parameters, <code>θ</code>.</p>
  
  <pre><code>function gradients(x) # x = one input data point or a batch of input data points
      # stuff to calculate gradients of each parameter
      # returns gradients of each parameter
  </code></pre>
  
  <p>I then want to be able to do something like the following.</p>
  
  <pre><code>grads = gradients(x)
  update!(opt, θ, grads)
  </code></pre>
  
  <p>My question is: What form/type does my <code>gradient(x)</code> function need to return in order to do <code>update!(opt, θ, grads)</code>, and how do I do this?</p>