Cast int32 to bfloat16 does not run on A100 GPU
@yufang67 
I was able to execute the given code on GPU without any problem on the Linux virtual machine using tf-nightly 2.13.0-dev20230220 and on colab using TF v2.11. Please find the gist of [2.11](https://colab.research.google.com/gist/tiruk007/64342dba197c2a9fc60c141d0327ea62/2_11.ipynb) and refer to the [screenshot](https://user-images.githubusercontent.com/111861663/220181799-2a4d4b82-112c-4608-921f-a95210473b43.png) for tf-nightly.

Thank you!
 @tiruk007 thanks for the response. 
But it seems the cast run on CPU: Executing op Cast in device /job:localhost/replica:0/task:0/device:CPU:0 @gaikwadrahul8 
Could you please look into this ?

Thank you! Hi, @yufang67 

Apologize for the delay and I was able to replicate the issue with `Tensorflow==2.11` on Google Colab with `TPU` and I think `bfloat16` runs on TPU and you'll have to use `bfloat16` datatype with `TPU` runtype and it should work so for your reference I have added [gist-file](https://colab.research.google.com/gist/gaikwadrahul8/e443a8c05d2f7419868da2a589c39103/-59728.ipynb) and it's working as expected and even you can check the similar issue [#59624](https://github.com/tensorflow/tensorflow/issues/59624)

If issue still persists please let us know or Could you please confirm if this issue is resolved for you ? Please feel free to close the issue if it is resolved ? Thank you! Hi @gaikwadrahul8 ,

thanks for responds. As A100 GPU supports also bfloat16, im trying to evaluate the whole model on it. Some layers are already support bfloat16 (https://github.com/tensorflow/tensorflow/issues/59050#issuecomment-1428862660), i would like to know if there is plan to support also this cast op ?

thanks,  Hi, @yufang67 

Apologize for the delay and I hope you checked this [comment](https://github.com/tensorflow/tensorflow/issues/59050#issuecomment-1402914100) and at the moment we don't have the exact estimated time of arrival, if you have any workaround or implementation to cast `int32` to `bfloat16` for `GPU`, PR will be welcomed from your end and if you don't have then developer team will review and validate this feature request and developer team will decide upon the priority whether they will work on this feature request or not. Thank you!  @gaikwadrahul8 Thanks. how to submit the request to the developer team ?  Hi, @yufang67

Thank you for the confirmation so it seems like you don't have workaround or implementation from your end then I'll escalate this issue to developer team as feature request and they'll take care of this issue. Thank you! Hi, @yufang67 

I think at the moment `bfloat16` only supports `Nvidia A100 GPU's` so could you please try to run your code on `Nvidia A100 GPU's` and please let us know whether is it working or not as expected? Thank you! Hi @gaikwadrahul8,
Yes, all my tests and reported issue is based on single A100 80G GPU. @yufang67, Thank you for the confirmation and this issue we'll consider as feature request 

Hi, @sachinprasadhs 

Could you please look into this issue ? Thank you! @trevor-m, can you look into this? Looks like we don't register cast kernels to or from bfloat16 for most other types. Hi @reedwm, I spent some time on this but the cast code is a bit confusing.
Using the macros, I keep getting duplicate registration errors or missing instantiations.

I also tried to explicitly only add int32, which compiles, but when running the script it still always uses the CPU. Any idea what I'm missing here: https://github.com/trevor-m/tensorflow/commit/ac9a6b134ac022dd520cf0c3a73a222f0824418d ? Running the script in the original post with your patch https://github.com/trevor-m/tensorflow/commit/ac9a6b134ac022dd520cf0c3a73a222f0824418d fixes the issue for me. Without your patch, last Cast log line is

```
2023-03-24 01:52:55.531962: I tensorflow/core/common_runtime/eager/execute.cc:1588] Executing op Cast in device /job:localhost/replica:0/task:0/device:CPU:0
```

With your patch, the last log Cast log line is:

```
2023-03-24 02:16:08.281971: I tensorflow/core/common_runtime/placer.cc:114] Cast: (Cast): /job:localhost/replica:0/task:0/device:GPU:0
``` Oh wait but other Cast ops are still on the CPU, taking a look... With your patch, the CPU casts are casts from float32 to bfloat16. Casts to/from int32 are on the GPU.

In the following program, all casts except the cast from float32 to bfloat16 (the third cast) are on the GPU.

```python
import tensorflow as tf

tf.debugging.set_log_device_placement(True)

x = tf.constant(1, dtype=tf.int32)
y = tf.cast(x, tf.bfloat16)
z = tf.cast(y, tf.int32)

x = tf.constant(1, dtype=tf.float32)
y = tf.cast(x, tf.bfloat16)
z = tf.cast(y, tf.float32)
```

If you're having trouble with duplicate registration errors or missing instantiations, you can share a patch and i can take a look. Thanks @reedwm! I was able to figure it out, I was originally confused because [DEFINE_ALL_FROM](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/cast_op_gpu.cu.cc#L39) and [DEFINE_ALL_TO_*](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/cast_op_gpu.cu.cc#L104) are actually the same and one of them mislabeled the input type... Let me know if you have any suggestions to clean things up in the PR.
<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Feature Request

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

TensorFlow version 2.13.0-dev20230215

### Custom Code

Yes

### OS Platform and Distribution

ubuntu 20.04

### Mobile device

_No response_

### Python version

3.8

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

11.8/8.6

### GPU model and memory

single A100 80G

### Current Behaviour?

```shell
when using tf.cast to cast tf.int32 tensor to tf.bfloat16 tensor, op run on GPU.

when i convert int32->bfloat16, it run on CPU.
when i convert int32->float32->bfloat16, it run on GPU. is it expected ?
```


### Standalone code to reproduce the issue

```shell
import tensorflow as tf
from tensorflow.keras import mixed_precision

tf.debugging.set_log_device_placement(True)
policy = mixed_precision.Policy('mixed_bfloat16')
print(policy.name)
mixed_precision.set_global_policy(policy)

class toy_layer(tf.keras.layers.Layer):
  def build(self, input_shape):
    self.kernel = self.add_weight('kernel', (input_shape[-1], 10))
  def call(self, inputs):
    out = tf.linalg.matmul(inputs, self.kernel)
    out2 = tf.ones((10, 10), dtype=tf.int32)
    #out2 = tf.cast(out2, tf.float32, name="cast_out2_1")
    out2 = tf.cast(out2, out.dtype, name="cast_out2_2")
    out3 = out * out2
    return out3

layer = toy_layer()
y = layer(tf.ones((10, 10)))
```


### Relevant log output

```shell
2023-02-17 16:26:34.748024: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-17 16:26:35.370677: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
mixed_bfloat16
2023-02-17 16:26:36.876507: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1635] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 78915 MB memory:  -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0001:00:00.0, compute capability: 8.0
input: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:36.886233: I tensorflow/core/common_runtime/placer.cc:114] input: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
_EagerConst: (_EagerConst): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:36.886254: I tensorflow/core/common_runtime/placer.cc:114] _EagerConst: (_EagerConst): /job:localhost/replica:0/task:0/device:GPU:0
output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:36.886267: I tensorflow/core/common_runtime/placer.cc:114] output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:36.889118: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
input: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.293473: I tensorflow/core/common_runtime/placer.cc:114] input: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
_EagerConst: (_EagerConst): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.293513: I tensorflow/core/common_runtime/placer.cc:114] _EagerConst: (_EagerConst): /job:localhost/replica:0/task:0/device:GPU:0
output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.293521: I tensorflow/core/common_runtime/placer.cc:114] output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.294777: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
dims: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.295227: I tensorflow/core/common_runtime/placer.cc:114] dims: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
value: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.295238: I tensorflow/core/common_runtime/placer.cc:114] value: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
Fill: (Fill): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.295250: I tensorflow/core/common_runtime/placer.cc:114] Fill: (Fill): /job:localhost/replica:0/task:0/device:GPU:0
output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.295257: I tensorflow/core/common_runtime/placer.cc:114] output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.295729: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Fill in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.297303: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.297667: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.297928: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.298227: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
seed: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.298533: I tensorflow/core/common_runtime/placer.cc:114] seed: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
StatelessRandomGetKeyCounter: (StatelessRandomGetKeyCounter): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.298547: I tensorflow/core/common_runtime/placer.cc:114] StatelessRandomGetKeyCounter: (StatelessRandomGetKeyCounter): /job:localhost/replica:0/task:0/device:GPU:0
key_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.298561: I tensorflow/core/common_runtime/placer.cc:114] key_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
counter_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.298573: I tensorflow/core/common_runtime/placer.cc:114] counter_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.299212: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op StatelessRandomGetKeyCounter in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.300704: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
shape: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.300963: I tensorflow/core/common_runtime/placer.cc:114] shape: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
key: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.300974: I tensorflow/core/common_runtime/placer.cc:114] key: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
counter: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.300979: I tensorflow/core/common_runtime/placer.cc:114] counter: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
alg: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.300993: I tensorflow/core/common_runtime/placer.cc:114] alg: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
StatelessRandomUniformV2: (StatelessRandomUniformV2): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.300999: I tensorflow/core/common_runtime/placer.cc:114] StatelessRandomUniformV2: (StatelessRandomUniformV2): /job:localhost/replica:0/task:0/device:GPU:0
output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.301008: I tensorflow/core/common_runtime/placer.cc:114] output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.301526: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op StatelessRandomUniformV2 in device /job:localhost/replica:0/task:0/device:GPU:0
x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.302304: I tensorflow/core/common_runtime/placer.cc:114] x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.302316: I tensorflow/core/common_runtime/placer.cc:114] y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
Sub: (Sub): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.302332: I tensorflow/core/common_runtime/placer.cc:114] Sub: (Sub): /job:localhost/replica:0/task:0/device:GPU:0
z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.302338: I tensorflow/core/common_runtime/placer.cc:114] z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.302706: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Sub in device /job:localhost/replica:0/task:0/device:GPU:0
x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.303361: I tensorflow/core/common_runtime/placer.cc:114] x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.303372: I tensorflow/core/common_runtime/placer.cc:114] y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
Mul: (Mul): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.303379: I tensorflow/core/common_runtime/placer.cc:114] Mul: (Mul): /job:localhost/replica:0/task:0/device:GPU:0
z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.303391: I tensorflow/core/common_runtime/placer.cc:114] z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.303748: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Mul in device /job:localhost/replica:0/task:0/device:GPU:0
x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.304205: I tensorflow/core/common_runtime/placer.cc:114] x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.304216: I tensorflow/core/common_runtime/placer.cc:114] y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
AddV2: (AddV2): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.304223: I tensorflow/core/common_runtime/placer.cc:114] AddV2: (AddV2): /job:localhost/replica:0/task:0/device:GPU:0
z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.304229: I tensorflow/core/common_runtime/placer.cc:114] z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.304663: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op AddV2 in device /job:localhost/replica:0/task:0/device:GPU:0
resource_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.305249: I tensorflow/core/common_runtime/placer.cc:114] resource_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
VarHandleOp: (VarHandleOp): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.305263: I tensorflow/core/common_runtime/placer.cc:114] VarHandleOp: (VarHandleOp): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.305637: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
resource: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.306104: I tensorflow/core/common_runtime/placer.cc:114] resource: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
value: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.306122: I tensorflow/core/common_runtime/placer.cc:114] value: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
AssignVariableOp: (AssignVariableOp): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.306139: I tensorflow/core/common_runtime/placer.cc:114] AssignVariableOp: (AssignVariableOp): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.306554: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op AssignVariableOp in device /job:localhost/replica:0/task:0/device:GPU:0
x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.307498: I tensorflow/core/common_runtime/placer.cc:114] x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
Cast: (Cast): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.307523: I tensorflow/core/common_runtime/placer.cc:114] Cast: (Cast): /job:localhost/replica:0/task:0/device:GPU:0
y_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.307532: I tensorflow/core/common_runtime/placer.cc:114] y_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.307943: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Cast in device /job:localhost/replica:0/task:0/device:GPU:0
resource: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.308714: I tensorflow/core/common_runtime/placer.cc:114] resource: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
ReadVariableOp: (ReadVariableOp): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.308734: I tensorflow/core/common_runtime/placer.cc:114] ReadVariableOp: (ReadVariableOp): /job:localhost/replica:0/task:0/device:GPU:0
value_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.308743: I tensorflow/core/common_runtime/placer.cc:114] value_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.309154: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op ReadVariableOp in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.309366: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Cast in device /job:localhost/replica:0/task:0/device:GPU:0
a: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.309693: I tensorflow/core/common_runtime/placer.cc:114] a: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
b: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.309710: I tensorflow/core/common_runtime/placer.cc:114] b: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
MatMul: (MatMul): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.309727: I tensorflow/core/common_runtime/placer.cc:114] MatMul: (MatMul): /job:localhost/replica:0/task:0/device:GPU:0
product_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.309738: I tensorflow/core/common_runtime/placer.cc:114] product_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.310165: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op MatMul in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.897703: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.897841: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op _EagerConst in device /job:localhost/replica:0/task:0/device:GPU:0
dims: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.898261: I tensorflow/core/common_runtime/placer.cc:114] dims: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
value: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.898272: I tensorflow/core/common_runtime/placer.cc:114] value: (_Arg): /job:localhost/replica:0/task:0/device:CPU:0
Fill: (Fill): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.898280: I tensorflow/core/common_runtime/placer.cc:114] Fill: (Fill): /job:localhost/replica:0/task:0/device:GPU:0
output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.898285: I tensorflow/core/common_runtime/placer.cc:114] output_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.898934: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Fill in device /job:localhost/replica:0/task:0/device:GPU:0
x: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.899509: I tensorflow/core/common_runtime/placer.cc:114] x: (_DeviceArg): /job:localhost/replica:0/task:0/device:CPU:0
Cast: (Cast): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.899523: I tensorflow/core/common_runtime/placer.cc:114] Cast: (Cast): /job:localhost/replica:0/task:0/device:CPU:0
y_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.899530: I tensorflow/core/common_runtime/placer.cc:114] y_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:CPU:0
2023-02-17 16:26:37.899935: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Cast in device /job:localhost/replica:0/task:0/device:CPU:0
x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.900335: I tensorflow/core/common_runtime/placer.cc:114] x: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.900348: I tensorflow/core/common_runtime/placer.cc:114] y: (_Arg): /job:localhost/replica:0/task:0/device:GPU:0
Mul: (Mul): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.900361: I tensorflow/core/common_runtime/placer.cc:114] Mul: (Mul): /job:localhost/replica:0/task:0/device:GPU:0
z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.900369: I tensorflow/core/common_runtime/placer.cc:114] z_RetVal: (_Retval): /job:localhost/replica:0/task:0/device:GPU:0
2023-02-17 16:26:37.901006: I tensorflow/core/common_runtime/eager/execute.cc:1514] Executing op Mul in device /job:localhost/replica:0/task:0/device:GPU:0
```
</details>