Abort when running tensorflow.python.eager.remote.connect_to_remote_host
@dmc1778 In TF v2.13 the session is crashing and in tf-nightly the session is not getting aborted. Could you please have a look at this [gist](https://colab.research.google.com/gist/sushreebarsa/c95f2b92e141f1fd7ba231149bce4780/61542.ipynb) and confirm?
Thank you! > @dmc1778 In TF v2.13 the session is crashing and in tf-nightly the session is not getting aborted. Could you please have a look at this [gist](https://colab.research.google.com/gist/sushreebarsa/c95f2b92e141f1fd7ba231149bce4780/61542.ipynb) and confirm? Thank you!

That is correct. @dmc1778 Thank you for the confirmation. As the issue is not appearing in tf-nightly then it would be fixed in next release so could you please let us know if it is good to close the ticket?
Thank you! > @dmc1778 Thank you for the confirmation. As the issue is not appearing in tf-nightly then it would be fixed in next release so could you please let us know if it is good to close the ticket? Thank you!

I think it is safe to close this issue. @dmc1778 Thank you for the confirmation. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61542">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61542">No</a>

### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

binary

### TensorFlow version

2.11.0

### Custom code

Yes

### OS platform and distribution

22.04

### Mobile device

_No response_

### Python version

3.9

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

nvidia-cudnn-cu11==8.6.0.163, cudatoolkit=11.8.0

### GPU model and memory

_No response_

### Current behavior?

NaN string argument

### Standalone code to reproduce the issue

```shell
import tensorflow as tf
import numpy as np
from tensorflow.python.eager import remote
try:
  try:
    with tf.device('/CPU'):
      arg_0 = "nan"
      out = remote.connect_to_remote_host(arg_0,)
  except Exception as e:
    print("Error:"+str(e))
  try:
    with tf.device('/GPU:0'):
      remote.connect_to_remote_host(arg_0,)
  except Exception as e:
    print("Error:"+str(e))
except Exception as e:
  print("Error:"+str(e))
```
```


### Relevant log output

```shell
2023-08-13 01:22:37.499369: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-08-13 01:22:38.459392: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.480510: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.480708: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.481081: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-13 01:22:38.481707: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.481844: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.481961: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.536637: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.536859: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.536991: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-08-13 01:22:38.537094: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1725 MB memory:  -> device: 0, name: NVIDIA GeForce GTX 1660 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5
2023-08-13 01:22:38.546718: E tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:589] INVALID_ARGUMENT: Could not interpret "nan" as a host-port pair.
E0813 01:22:38.546961566 1686085 completion_queue.cc:244]    assertion failed: queue.num_items() == 0
Aborted

```
```
