title: Modify Tensorflow (Keras) Optimizer (for Layerwise Learning Rate Multipliers)
body: <p>I am trying to train a CNN in tensorflow (keras) with different learning rates per layer. As this option is not included in tensorflow i am trying to modify an already existing optimizer like suggested in this <a href="https://github.com/keras-team/keras/issues/5920#issuecomment-328890905" rel="nofollow noreferrer">github comment</a>.</p>
  
  <p>When I simply copy the source code of the SGD optimizer that i found in </p>
  
  <blockquote>
    <p>"C:\Users\user\Anaconda3\envs\tf_gpu\Lib\site-packages\tensorflow_core\python\keras\optimizers.py"</p>
  </blockquote>
  
  <p>(without any modifications) into my script and I try to run it I get the following error: </p>
  
  <pre><code>TypeError                                 Traceback (most recent call last)
  &lt;ipython-input-2-757ea125a090&gt; in &lt;module&gt;
  ----&gt; 1 opti = SGD(lr=0.001, momentum=0.9, nesterov=False)
  
  &lt;ipython-input-1-dad895e21911&gt; in __init__(self, lr, momentum, decay, nesterov, **kwargs)
       63 
       64   def __init__(self, lr=0.01, momentum=0., decay=0., nesterov=False, **kwargs):
  ---&gt; 65     super(SGD, self).__init__(**kwargs)
       66     with K.name_scope(self.__class__.__name__):
       67       self.iterations = K.variable(0, dtype='int64', name='iterations')
  
  TypeError: __init__() missing 1 required positional argument: 'name'
  </code></pre>
  
  <p>Do I need to make modifications directly in the source code and compile tensorflow from source or am I just doing something wrong? </p>
  
  <p>After attempting to install tensorflow (with GPU support) and getting a lot of errors with CuDNN and CUDA I ended up installing it using</p>
  
  <pre><code>conda install tensorflow-gpu 
  </code></pre>
  
  <p>which worked perfectly, so I am a bit scared to uninstall the now running tensorflow version and compile it from source.</p>
  
  <p>Also, if you know other ways how to modify the learning rate for each layer individually using tensorflow please let me know.</p>