title: Tensorflow, tf.reshape causes "Gradients do not exist for variables"
body: <p>I'm writing a fully connected layer using Tensorflow/Keras (TF version 2.1, Python 3.7 on Windows), but I've found that if I reshape my weights tensor before multiplying by it then Tensorflow doesn't seem to be able to calculate the gradient even if I just reshape to its own shape. Consider the following layer code:</p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  
  class FCLayer(tf.keras.layers.Layer):
      def __init__(self,output_size,cause_error = False):
          super(FCLayer,self).__init__()
          self.output_size = output_size
          self.cause_error = cause_error
  
      def build(self,input_shape): 
          self.input_size = input_shape[1]        
          weights = self.add_weight(shape=(self.input_size,
                                           self.output_size),
                                   initializer='random_normal',
                                   trainable=True)
  
          if self.cause_error:
              self.weights2 = tf.reshape( weights,
                                          shape = (self.input_size,
                                                   self.output_size))
          else:
              self.weights2 = weights
  
      def call(self, inputs):
          return tf.matmul(inputs, self.weights2)    
  </code></pre>
  
  <p>If this is used with cause_error = True, then I get the following output when training on mnist for 4 epochs (specific training code included below):</p>
  
  <pre><code>Train on 60000 samples, validate on 10000 samples
  Epoch 1/4
  WARNING:tensorflow:Gradients do not exist for variables ['sequential/dummy_layer/Variable:0'] when minimizing the loss.
  WARNING:tensorflow:Gradients do not exist for variables ['sequential/dummy_layer/Variable:0'] when minimizing the loss.
  60000/60000 [==============================] - 1s 20us/sample - loss: 2.4131 - accuracy: 0.0722 - val_loss: 2.3963 - val_accuracy: 0.0834
  Epoch 2/4
  60000/60000 [==============================] - 1s 12us/sample - loss: 2.4122 - accuracy: 0.0722 - val_loss: 2.3953 - val_accuracy: 0.0836
  Epoch 3/4
  60000/60000 [==============================] - 1s 12us/sample - loss: 2.4112 - accuracy: 0.0724 - val_loss: 2.3944 - val_accuracy: 0.0838
  Epoch 4/4
  60000/60000 [==============================] - 1s 13us/sample - loss: 2.4102 - accuracy: 0.0725 - val_loss: 2.3933 - val_accuracy: 0.0839
  </code></pre>
  
  <p>This is just a warning, but it is clear that the model is not really improving and obviously it needs those gradients.</p>
  
  <p>If I set cause_error=False I instead get the expected output (no warnings, modest improvements):</p>
  
  <pre><code>Train on 60000 samples, validate on 10000 samples
  Epoch 1/4
  60000/60000 [==============================] - 1s 16us/sample - loss: 2.3671 - accuracy: 0.1527 - val_loss: 2.3445 - val_accuracy: 0.1508
  Epoch 2/4
  60000/60000 [==============================] - 1s 12us/sample - loss: 2.3293 - accuracy: 0.1596 - val_loss: 2.3072 - val_accuracy: 0.1610
  Epoch 3/4
  60000/60000 [==============================] - 1s 13us/sample - loss: 2.2939 - accuracy: 0.1683 - val_loss: 2.2722 - val_accuracy: 0.1720
  Epoch 4/4
  60000/60000 [==============================] - 1s 13us/sample - loss: 2.2609 - accuracy: 0.1784 - val_loss: 2.2397 - val_accuracy: 0.1847
  </code></pre>
  
  <p>I suspect I need to somehow tell Tensorflow to keep track of the gradients, but am not quite sure how. It seems to do it automatically when I use tf.matmul, and I'm pretty sure this kind of code used to work in TF 1.</p>
  
  <p>The specific code I used to execute was (adapted from the mnist tutorial):</p>
  
  <pre><code>batch_size = 128
  num_classes = 10
  epochs = 4
  
  # input image dimensions
  img_rows, img_cols = 28, 28
  
  # the data, split between train and test sets
  (x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
  
  
  x_train = x_train.reshape(x_train.shape[0], img_rows* img_cols)
  x_test = x_test.reshape(x_test.shape[0], img_rows*img_cols)
  input_shape = (img_rows * img_cols)
  
  x_train = x_train.astype('float32')
  x_test = x_test.astype('float32')
  x_train /= 255
  x_test /= 255
  print('x_train shape:', x_train.shape)
  print(x_train.shape[0], 'train samples')
  print(x_test.shape[0], 'test samples')
  
  # convert class vectors to binary class matrices
  y_train = tf.keras.utils.to_categorical(y_train, num_classes)
  y_test = tf.keras.utils.to_categorical(y_test, num_classes)
  
  model = tf.keras.models.Sequential()
  
  dummy_layer = FCLayer(10, cause_error = True)
  model.add( dummy_layer )
  model.add( tf.keras.layers.Dense(10, activation='softmax') )
  
  model.compile(loss=tf.keras.losses.categorical_crossentropy,
                optimizer=tf.keras.optimizers.Adadelta(),
                metrics=['accuracy'])
  
  model.fit(x_train, y_train,
            batch_size=batch_size,
            epochs=epochs,
            verbose=1,
            validation_data=(x_test, y_test))
  </code></pre>