TensorFlow int32 data type memory copy issue
Hi, just to check if any updates for this issue Anyone checked this PR? Any update?
### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

binary

### TensorFlow version

TF2.15

### Custom code

Yes

### OS platform and distribution

Linux Ubunto 22.04

### Mobile device

_No response_

### Python version

3.10

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

12.3

### GPU model and memory

_No response_

### Current behavior?

On eager mode, when doing CopyCPUTensorToGPU, if input is a int32 tensor(allocated on device) and hold by an _Arg op on device, this int32 device tensor will be treated as a host src, here is log to prove that behavior(see host_src=0x7f7351400500, which is a device mem pointer):
38974 2023-11-30 08:18:14.566732: I tensorflow/core/common_runtime/copy_tensor.cc:211] Copy edge_2_tensor
38975 2023-11-30 08:18:14.566740: I tensorflow/core/common_runtime/gpu/gpu_util.cc:315] CopyCPUTensorToGPU
38976 2023-11-30 08:18:14.566750: I tensorflow/compiler/xla/stream_executor/stream.cc:1030] [stream=0x7e88fe0,impl=0x83b6310] Called Stream::ThenWaitFor(other=0x7e89090)
38977 2023-11-30 08:18:14.566765: I tensorflow/compiler/xla/stream_executor/stream.cc:2002] [stream=0x7e88fe0,impl=0x83b6310] Called Stream::ThenMemcpy(gpu_dst=0x7f7351400600, host_src=**0x7f7351400500**, size=4)

However, this case still can pass even with this abnormal behavior. But when integrating Next Pluggable Device with PJRT buffer, a device pointer being treated as host src would cause critical value issues.



### Standalone code to reproduce the issue

```shell
https://github.com/horovod/horovod/blob/7e4d99386ee4196d767943e30945e45640049ebe/test/parallel/test_tensorflow.py#L3130
Note: this case can pass, only for reproducing the abnormal behavior.
```


### Relevant log output

```shell
38974 2023-11-30 08:18:14.566732: I tensorflow/core/common_runtime/copy_tensor.cc:211] Copy edge_2_tensor
38975 2023-11-30 08:18:14.566740: I tensorflow/core/common_runtime/gpu/gpu_util.cc:315] CopyCPUTensorToGPU
38976 2023-11-30 08:18:14.566750: I tensorflow/compiler/xla/stream_executor/stream.cc:1030] [stream=0x7e88fe0,impl=0x83b6310] Called Stream::ThenWaitFor(other=0x7e89090)
38977 2023-11-30 08:18:14.566765: I tensorflow/compiler/xla/stream_executor/stream.cc:2002] [stream=0x7e88fe0,impl=0x83b6310] Called Stream::ThenMemcpy(gpu_dst=0x7f7351400600, host_src=**0x7f7351400500**, size=4)
```
