Inconsistent heoretical and numeric gradient for tf.math.floor
Hi @drewshark,

I was able to replicate the issue in Google Colab using Tensorflow 2.10, 2.12 and tf-nightly(2.14.0.dev20230510). Please find the gists- [TF v2.10](https://colab.sandbox.google.com/gist/synandi/bd080f93b862f9680ba3fdaa51b15335/60556_2-10.ipynb), [TF v2.12](https://colab.sandbox.google.com/gist/synandi/102cb5ec11d6e35ae928852af5764ca6/60556_2-12.ipynb) & [TF-nightly](https://colab.sandbox.google.com/gist/synandi/57e9b3b50464b102d84361d106aedd7a/60556_tfnightly.ipynb). Thank you! from inconsistent you mean when **``` x = [2.] ```** or **``` x = [4.]```** getting numerical jacobians as **```array([[512.]], dtype=float32)```** ,am i right? Hi @Shivachauhan17 , yes this is my observation. Since the numerical Jacobins are sometime considered as the ground truth when testing gradient computations, it would be nice if numerical jacobians are also array([[0.]]). okay, i will look at the implemtation  I noticed that a few other functions suffer from similar issues, this includes `tf.math.ceil`, `tf.experimental.numpy.ceil`, `tf.math.round`, `tf.math.rint`, `tf.experimental.numpy.heaviside`.

See the [gist here](https://colab.research.google.com/drive/1G_3GnYqSxIvLRTNB99YjqwE_kBcE9mB_?usp=sharing). This is an artifact of how the "numerical" gradient is computed.  It uses central differences, so evaluates
```
df/dx = [f(x + delta) - f(x - delta)] / (2*delta)
```
Floor (and the other functions you mention) have a discontinuous gap at whole numbers.  So it evaluates
```
df/dx(4.0) = f(4.0 + delta) - f(4.0 - delta) / (2 * delta)
                 = (4.0 - 3.0) / (2 * delta)    // for any delta < 1
```
It uses a default delta of 1.0/1024, so you're getting  `1.0 / (2.0 / 1024) = 512`.  The theoretical gradient is more correct, since the derivative is zero at either side of the discontinuity. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60556">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60556">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

2.10.0

### Custom Code

Yes

### OS Platform and Distribution

_No response_

### Mobile device

_No response_

### Python version

3.9

### Bazel version

N/A

### GCC/Compiler version

N/A

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

When input is `tf.constant([4.])`, the theoretical gradient and numerical gradient of tf.math.floor are inconsistent. Same applies for tf.experimental.numpy.floor.

Expected behavior of tf.math.floor and tf.experimental.numpy.floor: consistent result between theoretical gradient and numerical gradient.

Interestingly, if I set `x = tf.constant([1.8])`, this issue does not happen. If I set `x=tf.constant([2.])`, this issue still occurs.


### Standalone code to reproduce the issue

```shell
import tensorflow as tf
x = tf.constant([4.])
th, nu= tf.test.compute_gradient(tf.math.floor, [x])
print(th, nu)  # (array([[0.]], dtype=float32),) (array([[512.]], dtype=float32),)
th, nu= tf.test.compute_gradient(tf.experimental.numpy.floor, [x])
print(th, nu)  # (array([[0.]], dtype=float32),) (array([[512.]], dtype=float32),)
```


### Relevant log output

_No response_</details>