`nan` while doing inference in C++ with half precision
 @p3achyjr  Thanks for reporting this issue.

Looks like inputs are not being provided to the model's session.  Also, the minimal reproducible code for the issue is incomplete. 

Can you please provide the steps you have followed and also did you try setting up the [dtype policy](https://www.tensorflow.org/guide/mixed_precision#setting_the_dtype_policy) to mixed_float16.

Thanks.
 Thanks for getting back--the dtype policy is set to mixed_float16 in python. Not sure how to do that in c++.

I also ended up solving this as it turns out my tensors were not zero-initialized. I wonder if there's a way to do that automatically?

Also let me know if there is a better way to do half precision in C++, as right now I need to construct and run a graph to cast my tensors from float -> half -> back to float every time I want to call into my model. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59982">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59982">No</a>
 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. Hi, Could you please check the `ZerosLike` function https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/zeros-like which returns tensor of zeros.
For half precision in C++, to the best of knowledge I can think of `Requantize` OP https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/requantize, check if that helps you. Thanks! ```
using namespace tensorflow;

const static std::vector<std::string> kInputNames = {
    "serving_default_args_0:0",
};

const static std::vector<std::string> kOutputNames = {
    "StatefulPartitionedCall:0",
};

int main(int argc, char** argv) {
  SavedModelBundleLite model_bundle_;
  LoadSavedModel(SessionOptions(), RunOptions(), "/tmp/model",
                 {"serve"}, &model_bundle_);

  Tensor input_features(DataType::DT_HALF,
                        {1, 8, 8});
  input_features.flat_half().setZero(); // Initialize to zero
  std::vector<std::pair<std::string, Tensor>> inputs{
      std::make_pair(kInputNames[0], input_features),
  };

  std::vector<Tensor> output_buf = {
      Tensor(DataType::DT_FLOAT, {1, 6, 6}),
  };

  // Run inference with mixed precision
  Status status = model_bundle_.GetSession()->Run(
      {ops::Cast(scope, input_features, DataType::DT_HALF)},
      {kOutputNames[0]}, {}, &output_buf);

  if (!status.ok()) {
    LOG(ERROR) << "Inference failed: " << status;
    return -1;
  }

  // Print the output tensor values for debugging
  auto output_flat = output_buf[0].flat<float>();
  for (int i = 0; i < output_flat.size(); i++) {
    LOG(INFO) << "Output[" << i << "] = " << output_flat(i);
  }

  return 0;
}
```

Note that the code now initializes the **input_features** tensor values to zero using the **setZero()** method on the **flat_half()** accessor. The input tensor is also now created with **DataType::DT_HALF**, and the output tensor is created with **DataType::DT_FLOAT** to match the output of the Python model.

During inference, the code now uses mixed precision by casting the input tensor to half-precision using **ops::Cast(scope, input_features, DataType::DT_HALF)**. The output tensor is also created as a **std::vector<Tensor>** so that the Run() method can fill in the output values.
After running inference, the code prints out the values of the output tensor for debugging purposes. Thanks all! I think we can close this :) Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59982">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59982">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

No

### Source

source

### Tensorflow Version

2.11

### Custom Code

Yes

### OS Platform and Distribution

Debian GNU/Linux 10

### Mobile device

_No response_

### Python version

3.7

### Bazel version

6.1.0

### GCC/Compiler version

LLVM 12

### CUDA/cuDNN version

11.6

### GPU model and memory

Tesla T4

### Current Behaviour?

```shell
I've trained a model in python using mixed precision. However, when loading the model in C++ and trying to perform inference, The output is all `nan`.

I've tested loading the model in python and it works.

Some ideas might be:

- Toolchain is different.
- Tensors in C++ are not zero-initialized.
```


### Standalone code to reproduce the issue

```shell
Python


class Model(tf.keras.Model):
  def __init__(self):
    self.conv = tf.keras.layers.Conv2D(
        128,
        activation='relu')

  def call(self, x):
    return self.conv(x)

x = tf.convert_to_tensor(np.random.random(1, 8, 8))
model = Model()
model(x)
model.save('/tmp/model')
```

C++
```
using namespace ::tensorflow;

const static std::vector<std::string> kInputNames = {
    "serving_default_args_0:0",
};

const static std::vector<std::string> kOutputNames = {
    "StatefulPartitionedCall:0",
};

int main(int argc, char** argv) {
  SavedModelBundleLite model_;
  LoadSavedModel(SessionOptions(), RunOptions(), "/tmp/model",
                 {"serve"}, &model_bundle_);

  Tensor input_features(DataType::DT_FLOAT,
                        {1, 8, 8});
  std::vector<Tensor> cast_output;
  Status status =
      session.Run({ops::Cast(scope, input_features, DataType::DT_HALF),
                   ops::Cast(scope, input_state, DataType::DT_HALF)},
                  &cast_output);
  std::vector<std::pair<std::string, Tensor>>{
      std::make_pair(kInputNames[0], cast_output[0]),
  };

  std::vector<Tensor> output_buf = {
      Tensor(DataType::DT_HALF, {1, 6, 6}),
  };

  // this is nan
  model.GetSession()->Run(inputs, output_names, {}, output_buf);
}
```
```


### Relevant log output

```shell
2023-03-14 05:08:59.641765: I cc/scripts/play_model.cc:94] -nan
2023-03-14 05:08:59.641818: I cc/scripts/play_model.cc:95] -nan
2023-03-14 05:08:59.655568: I cc/scripts/play_model.cc:119] ------- Model Stats -------
2023-03-14 05:08:59.655609: I cc/scripts/play_model.cc:120] Top Move: Loc(0, 0)
2023-03-14 05:08:59.655621: I cc/scripts/play_model.cc:121] Win: -nan Loss: -nan
2023-03-14 05:08:59.655677: I cc/scripts/play_model.cc:123] -----Board-----
0  ○ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
1  ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
2  ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
3  ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅
4  ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
5  ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
6  ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
7  ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
8  ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
9  ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅
10 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
11 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
12 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
13 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
14 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
15 ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅ ⋅ ⋅ + ⋅ ⋅ ⋅
16 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
17 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
18 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅
   A B C D E F G H I J K L M N O P Q R S
```
</details>