Error: An operation has `None` for gradient with categorical_crossentropy I am trying to train my discriminator network using Keras with TensorFlow backend. 
The network is meant to classify the input into one of the 9 output labels. I am passing a 2D input (height, width, no channels) and a one-hot vector for the output. I was able to train the network independently using fit(). However, now that I have switched to train_on_batch, it is giving me the error mentioned.

This is my discriminator code:

def build_discriminator(time_steps, feature_size, input_spectrogram=None):
    spectrogram = Input(shape=(time_steps, feature_size))
    # spectrogram = tf.placeholder(tf.float32, shape=(None, time_steps, feature_size))
    layer0 = Reshape((time_steps, feature_size, 1))(spectrogram)
    layer1 = Conv2D(32, kernel_size=(3,3), padding='same')(layer0)
    #model.add(LeakyReLU(alpha=0.01))
    layer2 = MaxPooling2D(pool_size=(4,4))(layer1)

    layer3 = Conv2D(16, kernel_size=(3,3), padding='same')(layer2)
    #model.add(LeakyReLU(alpha=0.05))
    layer4 = MaxPooling2D(pool_size=(4,4))(layer3)

    layer5 = Conv2D(16, kernel_size=(3,3), padding='same')(layer4)
    #model.add(LeakyReLU(alpha=0.05))
    layer6 = MaxPooling2D(pool_size=(4,4))(layer5)

    layer7 = Flatten()(layer6)
    layer8 = Dense(16)(layer7)
    prediction = Dense(9, activation = 'softmax')(layer8)
    # prediction = Dropout(0.1)(layer9)

    model = Model(spectrogram, prediction)

    opt = optimizers.Adam(lr=0.002, beta_1=0.5)
    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])

    return model


This is the code that trains the discriminator:

    x_real = batch_x[:half_batch, :, :]
    labels_real = batch_labels[:half_batch]
    d_loss1, _ = discriminator.train_on_batch(x_real, to_categorical(labels_real, num_classes=9))
    # generate 'fake' examples
    x_fake, labels_fake = generate_fake_samples(batch_x[half_batch:, :, :], batch_labels[half_batch:], generator)
    # update discriminator model weights
    d_loss2, _ = discriminator.train_on_batch(x_fake, to_categorical(labels_fake, num_classes=9))
    # update the generator via the discriminator's error
    g_loss, acc = gan.train_on_batch([batch_x, batch_targets], to_categorical(batch_targets, num_classes=9))


It is throwing the error on this line:

d_loss1, _ = discriminator.train_on_batch(x_real, to_categorical(labels_real, num_classes=9))


The error traceback:

  File "gan.py", line 126, in train
    d_loss1, _ = discriminator.train_on_batch(x_real, to_categorical(labels_real, num_classes=9))
  File "/home/pallavi/anaconda3/lib/python3.7/site-packages/keras/engine/training.py", line 1513, in train_on_batch
    self._make_train_function()
  File "/home/pallavi/anaconda3/lib/python3.7/site-packages/keras/engine/training.py", line 316, in _make_train_function
    loss=self.total_loss)
  File "/home/pallavi/anaconda3/lib/python3.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/pallavi/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 75, in symbolic_fn_wrapper
    return func(*args, **kwargs)
  File "/home/pallavi/anaconda3/lib/python3.7/site-packages/keras/optimizers.py", line 504, in get_updates
    grads = self.get_gradients(loss, params)
  File "/home/pallavi/anaconda3/lib/python3.7/site-packages/keras/optimizers.py", line 93, in get_gradients
    raise ValueError('An operation has `None` for gradient. '
ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.


I am using tensorflow-gpu 2.0.0 with keras 2.3.1.
Could someone please help me understand where I am going wrong? Thanks is advance!
This link might be helpful to you. you might be missing something in your code.

https://github.com/keras-team/keras/issues/12521