mean_absolute_error outputs weird results when input is nan on GPU
Hi @trickiwoo. Thanks for reporting this issue.

I was able to reproduce this in 2.11 and TF Nightly 2.13.0-dev20230301. Please find the gist [here](https://colab.research.google.com/gist/pjpratik/8cc469edd8c5b0fc13783e2a8bcefabb/59838.ipynb#scrollTo=3Wms-TQ7ExEI).

@SuryanarayanaY Could you please look into this. Thank you. Hi @trickiwoo , Thanks for your time for reporting this issue.

Basically the issue is coming due to tf.cast which is called by this API. With CPU `tf.cast(np.nan, tf.int32)` returns the min value of `tf.int32` i.e `-2147483648`. But with GPU  `tf.cast(np.nan, tf.int32)` casting to `'0'`. This may be due to inconsistent behaviour with `nan` which is an undefined value. The result is coming from `pywrap_tfe.TFE_Py_FastPathExecute`. This type of behaviour expected with garbage data.

 Thanks for getting back. It would be great if there could be some warnings or exceptions here for the nan inputs Casting `NaN` to int is undefined behavior, since it is not representable in the destination type.  We do not guarantee any particular results in this case, nor consistency between CPU and GPU - essentially you are giving it garbage data, so you will get garbage data out.  Casts are used everywhere, and it's essential they operate quickly, so we do not do any bounds checking on casts - if there's a chance the input will be out-of-range, you need to do a saturated case or explicit edge-case handling yourself. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59838">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59838">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

2.13.0-dev20230222

### Custom Code

Yes

### OS Platform and Distribution

_No response_

### Mobile device

_No response_

### Python version

_No response_

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

8.2.4

### GPU model and memory

_No response_

### Current Behaviour?

```shell
mean_absolute_error outputs weird results when input is nan on GPU
```


### Standalone code to reproduce the issue

```shell
import tensorflow as tf
import numpy as np

labels = [np.nan]
predictions = [1]
mae = tf.keras.metrics.mean_absolute_error(labels, predictions)
print(mae)
```


### Relevant log output

```shell
tf.Tensor(1, shape=(), dtype=int32)
```
</details>