Gradient Error (No gradient defined for operation 'bilateral_layer_1/Lu_10' (op type : Lu)) in CRFLayer
Hi @rishabh316 ,

It might be possible that any of the Op may not have gradient implementation and you are trying to calculate the gradient for it. Please cross check it once. I am attaching the lost of [raw_ops](https://www.tensorflow.org/api_docs/python/tf/raw_ops) in tensorflow with the status of gradient. Please cross check whether any of your Op falling under the category of No Gradient implementation.

Also [tf.gradients](https://www.tensorflow.org/api_docs/python/tf/gradients) valid only in graph context. I recommend to use [tf.GradientTape ](https://www.tensorflow.org/api_docs/python/tf/GradientTape) instead which is best suitable for Tf2.x versions.

Thanks! > Hi @rishabh316 ,
> 
> It might be possible that any of the Op may not have gradient implementation and you are trying to calculate the gradient for it. Please cross check it once. I am attaching the lost of [raw_ops](https://www.tensorflow.org/api_docs/python/tf/raw_ops) in tensorflow with the status of gradient. Please cross check whether any of your Op falling under the category of No Gradient implementation.
> 
> Also [tf.gradients](https://www.tensorflow.org/api_docs/python/tf/gradients) valid only in graph context. I recommend to use [tf.GradientTape ](https://www.tensorflow.org/api_docs/python/tf/GradientTape) instead which is best suitable for Tf2.x versions.
> 
> Thanks!

Thanks for your response and suggestions sir.
Is there any way i can apply this CRFLayer by doing some modifications to the model. My issue got solved as i defined my custom gradient for that layer whose op type does not have pre-defined gradient given by tensorflow. Thanks for your suggestion. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61106">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61106">No</a>

The issue is from the new refined UNet v3.0 which i try to train my dataset on.
the link to the repo of that model is here:
[https://github.com/92xianshen/refined-unet-v3](url)

I am working on this code on Jupyter notebook for my convenience.
I design the Unet architecture code based on the architecture given in this UNet.py file of repo.

I am training this model on the data of pets dataset (oxford-iiit-pets dataset) whose link is given below:
[https://www.kaggle.com/datasets/tanlikesmath/the-oxfordiiit-pet-dataset](url)

The data from the dataset is categorized into training and validation dataset already.
The Jupyter Notebook named Fresh_Unet3 is attached.

The problem is that the whole code runs successfully and model also got compiled, model summary obtained, but when i try to fit the model, it gives error which is:

StagingError: in user code:

    File "c:\users\dell\appdata\local\programs\python\python39\lib\site-packages\keras\engine\training.py", line 1284, in train_function  *
        return step_function(self, iterator)  #here
    File "c:\users\dell\appdata\local\programs\python\python39\lib\site-packages\keras\engine\training.py", line 1268, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))  #here
    File "c:\users\dell\appdata\local\programs\python\python39\lib\site-packages\keras\engine\training.py", line 1249, in run_step  **
        outputs = model.train_step(data)  #here
    File "c:\users\dell\appdata\local\programs\python\python39\lib\site-packages\keras\engine\training.py", line 1054, in train_step
        self.optimizer.minimize(loss, self.trainable_variables, tape=tape)  #here
    File "c:\users\dell\appdata\local\programs\python\python39\lib\site-packages\keras\optimizers\optimizer.py", line 542, in minimize
        grads_and_vars = self.compute_gradients(loss, var_list, tape)  #here
    File "c:\users\dell\appdata\local\programs\python\python39\lib\site-packages\keras\optimizers\optimizer.py", line 275, in compute_gradients
        grads = tape.gradient(loss, var_list)  #here

    LookupError: No gradient defined for operation'bilateral_layer_1/Lu_10' (op type: Lu). In general every operation must have an associated `@tf.RegisterGradient` for correct autodiff, which this op is lacking. If you want to pretend this operation is a constant in your program, you may insert `tf.stop_gradient`. This can be useful to silence the error in cases where you know gradients are not needed, e.g. the forward pass of tf.custom_gradient. Please see more details in https://www.tensorflow.org/api_docs/python/tf/custom_gradient.

The library versions are as follows:
python = 3.9.6
tensorflow = '2.12.0'
keras = '2.12.0'

I am attaching zip file containing Jupyter Notebook of the file i'm working on and a Python file named CRFLayer.py which contains the code of CRF.
[UNET_MODELv3.zip](https://github.com/tensorflow/tensorflow/files/11893789/UNET_MODELv3.zip)

According to me, the error could be in the BilateralLayer class inside CRFLayer.py file.
Please provide your valuable suggestions and solution to this problem.