Is there some way to set the data type of the tflite model to int8 here?
Yes, it is possible to convert the data type of a TensorFlow Lite (TFLite) model to int8. One way to do this is to use the TensorFlow Lite Converter, which provides options for quantizing and converting the data type of a TFLite model.

To convert the data type of a TFLite model to int8, you can use the tf.lite.TFLiteConverter class in TensorFlow. Here's an example code snippet:

```sh
import tensorflow as tf

# Load the original TFLite model
model_path = 'original_model.tflite'
interpreter = tf.lite.Interpreter(model_path=model_path)
interpreter.allocate_tensors()

# Create a TFLite converter with the int8 quantization configuration
converter = tf.lite.TFLiteConverter.from_saved_model('saved_model')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]

# Convert the model to int8
tflite_model = converter.convert()

# Save the converted model to a file
with open('int8_model.tflite', 'wb') as f:
    f.write(tflite_model)
```
In this example, the original TFLite model is loaded using the tf.lite.Interpreter class, and a new TFLite converter is created with the int8 quantization configuration. The optimizations attribute is set to [tf.lite.Optimize.DEFAULT] to enable default optimizations, and the supported_ops attribute is set to [tf.lite.OpsSet.TFLITE_BUILTINS_INT8] to specify that the converter should use the int8 quantization scheme.

Finally, the convert() method of the TFLite converter is called to convert the model to int8, and the resulting model is saved to a file using the open() function.

Note that the conversion to int8 may cause some loss of precision, which can affect the accuracy of the model. It is recommended to test the converted model thoroughly to ensure that it performs adequately for your use case. I'll try it later.Thanks!

> Yes, it is possible to convert the data type of a TensorFlow Lite (TFLite) model to int8. One way to do this is to use the TensorFlow Lite Converter, which provides options for quantizing and converting the data type of a TFLite model.
I'll try it later.Thanks! > converter = tf.lite.TFLiteConverter.from_saved_model('saved_model')

When i run    `converter = tf.lite.TFLiteConverter.from_saved_model('saved_model')`
It says
```
OSError: SavedModel file does not exist at: saved_model/{saved_model.pbtxt|saved_model.pb}
```
Then I create saved_model.
```
$:pwd
/home/sen/saved_model
$:ls
saved_model.pb  saved_model.pbtxt
```
It says 
```
RuntimeError: MetaGraphDef associated with tags {'serve'} could not be found in SavedModel, with available tags '[]'. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: `saved_model_cli`.
``` The error message suggests that the SavedModel file you provided as input to the from_saved_model method does not contain the expected content. Specifically, the method is looking for a MetaGraphDef with the 'serve' tag, but it cannot find it in the SavedModel file you provided.

To investigate further, you can use the saved_model_cli command line tool to inspect the contents of your SavedModel file. Here is an example command you can use:

```sh
saved_model_cli show --dir /home/sen/saved_model --all
```
This will show you all the available tag-sets in the SavedModel file, and you can check if the 'serve' tag is present or not.

If the 'serve' tag is not present, it is likely that the SavedModel was not exported with the correct signature or signature key. You can try exporting the model again with the correct signature, or you can modify the signature_def_map argument in the from_saved_model method to point to the correct signature key.

Here is an example of how to specify the signature_def_map argument:

```sh
converter = tf.lite.TFLiteConverter.from_saved_model('saved_model', signature_def_map={'serving_default': input_signature})
```
Here, serving_default is the name of the signature key, and input_signature is the input signature for the model. You should replace these with the correct values for your model. One other question, the tflite file I downloaded only has a `'1?lite-format=tflite'`, do I need to do something else with it?

 The `resnet_v2_101_1_default_1` directory holds my model.There is only  `'1?lite-format=tflite' ` inside.The saved_model directory  was created by me and I use `touch` to create `saved_model.pbtxt|saved_model.pb`,I think  I did it wrong.Should I do something else on `'1?lite-format=tflite'`.
```
$:saved_model_cli show --dir resnet_v2_101_1_default_1  --all
2023-04-16 15:48:45.766142: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-04-16 15:48:45.887963: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-04-16 15:48:45.891985: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-04-16 15:48:45.892027: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-04-16 15:48:46.582788: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-04-16 15:48:46.582869: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-04-16 15:48:46.582883: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
``` > One other question, the tflite file I downloaded only has a `'1?lite-format=tflite'`, do I need to do something else with it?

This is the standard file format for models that have been optimized for use on mobile and embedded devices.

Once you have downloaded the .tflite file, you can use it directly in your mobile or embedded application without any further processing. You will need to load the model into your application and use it to perform inference on your input data.

To load the model into your application, you can use the TensorFlow Lite Interpreter API. The exact steps for loading the model and running inference will depend on the programming language and framework you are using in your application. > The `resnet_v2_101_1_default_1` directory holds my model.There is only `'1?lite-format=tflite' ` inside.The saved_model directory was created by me and I use `touch` to create `saved_model.pbtxt|saved_model.pb`,I think I did it wrong.Should I do something else on `'1?lite-format=tflite'`.
> 
> ```
> $:saved_model_cli show --dir resnet_v2_101_1_default_1  --all
> 2023-04-16 15:48:45.766142: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
> To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
> 2023-04-16 15:48:45.887963: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
> 2023-04-16 15:48:45.891985: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
> 2023-04-16 15:48:45.892027: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
> 2023-04-16 15:48:46.582788: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
> 2023-04-16 15:48:46.582869: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
> 2023-04-16 15:48:46.582883: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
> ```

Based on the information provided, it seems like you have saved your model in TensorFlow Lite format (.tflite) in the "resnet_v2_101_1_default_1" directory. However, you have manually created a "saved_model" directory and its associated files ("saved_model.pbtxt" and "saved_model.pb") in the same directory. This may have caused the files to be overwritten or not saved correctly.

If you intend to use the TensorFlow SavedModel format instead of the TensorFlow Lite format, you can save your model using the following code:

```sh
import tensorflow as tf

# Load your ResNet model
model = tf.keras.applications.ResNet101V2(weights="imagenet")

# Save the model in SavedModel format
tf.saved_model.save(model, "saved_model")
```
This will create a directory named "saved_model" with the SavedModel files inside it.

Alternatively, if you intend to use the TensorFlow Lite format, you can convert the SavedModel to TensorFlow Lite using the following code:

```sh
converter = tf.lite.TFLiteConverter.from_saved_model("saved_model")
tflite_model = converter.convert()

# Save the TensorFlow Lite model to a file
with open("model.tflite", "wb") as f:
    f.write(tflite_model)
```
This will create a file named "model.tflite" containing the TensorFlow Lite model.

Regarding the warnings related to missing libraries, it seems like TensorFlow is unable to load some GPU-related libraries because you are running on a CPU-only system. These warnings can be safely ignored if you do not intend to use GPU acceleration for your model. Sorry, I still can't solve this problem.Here is my current progress. the saved_model.pb and saved_model.pbtxt are ampty.
```
$ pwd 
/home/sen/resnet_v2_101_1_default_1
$ ls 
'1?lite-format=tflite'   saved_model.pb   saved_model.pbtxt 
```
Then I run python.
```
>>> model_path = '/home/sen/resnet_v2_101_1_default_1/1?lite-format=tflite'
>>> interpreter = tf.lite.Interpreter(model_path=model_path)
>>> interpreter.allocate_tensors() 
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
>>> converter = tf.lite.TFLiteConverter.from_saved_model('/home/sen/resnet_v2_101_1_default_1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sen/iree.venv/lib/python3.10/site-packages/tensorflow/lite/python/lite.py", line 1792, in from_saved_model
    saved_model = _load(saved_model_dir, tags)
  File "/home/sen/iree.venv/lib/python3.10/site-packages/tensorflow/python/saved_model/load.py", line 828, in load
    result = load_partial(export_dir, None, tags, options)["root"]
  File "/home/sen/iree.venv/lib/python3.10/site-packages/tensorflow/python/saved_model/load.py", line 977, in load_partial
    root = load_v1_in_v2.load(export_dir, tags)
  File "/home/sen/iree.venv/lib/python3.10/site-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 284, in load
    result = loader.load(tags=tags)
  File "/home/sen/iree.venv/lib/python3.10/site-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 209, in load
    meta_graph_def = self.get_meta_graph_def_from_tags(tags)
  File "/home/sen/iree.venv/lib/python3.10/site-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 88, in get_meta_graph_def_from_tags
    return super(_EagerSavedModelLoader, self).get_meta_graph_def_from_tags(
  File "/home/sen/iree.venv/lib/python3.10/site-packages/tensorflow/python/saved_model/loader_impl.py", line 391, in get_meta_graph_def_from_tags
    raise RuntimeError(
RuntimeError: MetaGraphDef associated with tags {'serve'} could not be found in SavedModel, with available tags '[]'. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: `saved_model_cli`.
>>> converter = tf.lite.TFLiteConverter.from_saved_model('saved_model', signature_def_map={'serving_default': input_signature})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'input_signature' is not defined
``` It seems that you are still facing issues with your TensorFlow Lite model conversion. Based on the error message you provided, it appears that the from_saved_model() function is unable to find the MetaGraphDef associated with the 'serve' tag in your SavedModel directory.

To troubleshoot this issue, you can try using the saved_model_cli command-line tool to inspect the available tags in your SavedModel directory. Here's an example command you can use:

```sh
saved_model_cli show --dir /home/sen/resnet_v2_101_1_default_1 --tag_set serve --all
```
This should display information about the available tags and their associated MetaGraphDef signatures in your SavedModel directory.

As for the NameError you encountered when running the from_saved_model() function with the signature_def_map parameter, it seems that the variable input_signature was not defined before it was used in the function call. You may need to define input_signature as a dictionary that maps the input tensor names to their corresponding shapes and types, like this:

```sh
input_signature = {
    'input_tensor_name': (batch_size, height, width, channels),
    # define additional inputs as needed
}
```
Make sure to replace the placeholder values (batch_size, height, width, channels) with the actual input shapes and types that your model expects.

I hope this helps you make progress in converting your TensorFlow Lite model. Let me know if you have any further questions or issues!
 The following command will not output anything
> ```shell
> saved_model_cli show --dir /home/sen/resnet_v2_101_1_default_1 --tag_set serve --all
> ```
An error will be reported, I think we are using a different version of tensorflow.
```
 converter = tf.lite.TFLiteConverter.from_saved_model('saved_model', signature_def_map={'serving_default': input_signature})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: TFLiteConverterV2.from_saved_model() got an unexpected keyword argument 'signature_def_map'
```
> ```shell
> input_signature = {
>     'input_tensor_name': (batch_size, height, width, channels),
>     # define additional inputs as needed
> }
> ```
This is my [model](https://hub.tensorflow.google.cn/tensorflow/lite-model/resnet_v2_101/1/default/1), I think you can try it.Thanks!

 
Hi @linuxlonelyeagle 

The TFLite has `full integer quantization`  which converts 32-bit floating-point numbers (such as weights and activation outputs) to the nearest 8-bit fixed-point numbers. We can enable that while converting the tensorflow/keras models to tflite model.

Please refer this [documentation](https://www.tensorflow.org/lite/performance/post_training_integer_quant) on post training integer quantization.

The [model](https://hub.tensorflow.google.cn/tensorflow/lite-model/resnet_v2_101/1/default/1) which you are trying is a tflite model and it seems to be missing. We can get `resnet_v2_101` from [TF hub](https://hub.tensorflow.google.cn/google/imagenet/resnet_v2_101/classification/5) and convert to TF Lite model with int8 quantization.

Please find the [gist](https://colab.research.google.com/gist/pjpratik/86968ff929acb61fe09c4af40d6cf6e7/60334.ipynb) here doing the same.

Thanks.
 > Hi @linuxlonelyeagle
> 
> The TFLite has `full integer quantization` which converts 32-bit floating-point numbers (such as weights and activation outputs) to the nearest 8-bit fixed-point numbers. We can enable that while converting the tensorflow/keras models to tflite model.
> 
> Please refer this [documentation](https://www.tensorflow.org/lite/performance/post_training_integer_quant) on post training integer quantization.
> 
> The [model](https://hub.tensorflow.google.cn/tensorflow/lite-model/resnet_v2_101/1/default/1) which you are trying is a tflite model and it seems to be missing. We can get `resnet_v2_101` from [TF hub](https://hub.tensorflow.google.cn/google/imagenet/resnet_v2_101/classification/5) and convert to TF Lite model with int8 quantization.
> 
> Please find the [gist](https://colab.research.google.com/gist/pjpratik/86968ff929acb61fe09c4af40d6cf6e7/60334.ipynb) here doing the same.
> 
> Thanks.

Thank you  for your reply!
 Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60334">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60334">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Others

### Have you reproduced the bug with TF nightly?

No

### Source

binary

### Tensorflow Version

2.11

### Custom Code

Yes

### OS Platform and Distribution

linux ubuntu 20

### Mobile device

linux ubuntu 20 

### Python version

3.10.6

### Bazel version

No bazel

### GCC/Compiler version

9.4

### CUDA/cuDNN version

No cuda 

### GPU model and memory

No Gpu

### Current Behaviour?

I'm new to tensorflow, I was doing mlir related work before, I'm trying to download a model to test what I'm doing, but I'm having a problem, the conv and matmul I implemented on mlir only support int8, I'd like to ask if there is a way to convert the data type of the tflite model here to int8.

### Standalone code to reproduce the issue

```shell
I just want to ask if there is a way to convert data types.
```


### Relevant log output

```shell
No log.
```
</details>