TensorFlow Lite Converter wraps unpack operator with dequantize/quantize
@willisacs-arm Did you try with the latest TF version 2.13 and let us know the outcome?
Have a look at [this](https://www.tensorflow.org/lite/performance/post_training_quantization) reference for more information on quantization. Thank you! Hi yes the behaviour is the same for all versions above 2.12 Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61323">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/61323">No</a>
 accidentally closed issue Hi @willisacs-arm 

I did observe that it does not add Quantize/Dequantize stubs for intermittent runs.

@pkgoogle Could you please look into this issue?

Please find the reproducible [gist](https://colab.research.google.com/gist/pjpratik/bcd8711087797fe3874bf56073cb27d5/61323.ipynb) in TF 2.13.

Thanks. Hi @willisacs-arm, so currently the quantization is unable to quantize the unpack operator, so it actually processes that op in the original bit size. Would you like a feature request to make it quantizable? Are you working with a system that requires 8-bit operations Only? Hi, just to reiterate, the quantization sometimes works already if one just tries it enough times, so this doesn't seem to require a new feature, unless the fact that the quantization sometimes works is the bug here. It also works in patches before 2.12.0. 

Yes I am working with a system that requires int 16 or int 8 activations.  Also if I understand it correctly the documentation says that the converter should throw an error if an operation cannot be quantized?

https://www.tensorflow.org/lite/performance/post_training_quantization#integer_only Hi @willisacs-arm,  yeah that documentation is incorrect, we have not finalized on intended behavior so we have not updated it to prevent further confusion. That being said, "the quantization sometimes works already if one just tries it enough times" doesn't sound good, is there any way you can provide us information on when it starts working? like... What is your model where this happens (especially which ops are included), and after how many times does it start working? If you can show us the converted state after each time, that'll also be great. Thanks! > Hi @willisacs-arm
> 
> I did observe that it does not add Quantize/Dequantize stubs for intermittent runs.
> 
> @pkgoogle Could you please look into this issue?
> 
> Please find the reproducible [gist](https://colab.research.google.com/gist/pjpratik/bcd8711087797fe3874bf56073cb27d5/61323.ipynb) in TF 2.13.
> 
> Thanks.

Ah alright, thanks. 

The model is the one that was provided in the first comment in this issue, it has an unpack/unstack operator. It has also been provided neatly by pjpratik in the gist quoted in this reply. To reproduce the issue, run the notebook/gist and then repeat the last 2 steps until a quantized operator without dequantize/quantize ops appear in the Netron window. 

There seems to just be a random chance every time one runs the converter to get a functioning quantized unstack operator. A speculation would be that there's a memory overwrite somewhere. The behaviour is present even when running the converter on a single core and while keeping the representative dataset constant. The model file to convert was also kept constant.  I was able to reproduce the somewhat random behavior after 7 attempts with the attached gist. @abattery, can you please take a look? Thanks. did this has solution? i got this promblem too
### System information

- Linux Ubuntu 20.04
- TensorFlow installed from: pip source
- TensorFlow versions: 2.12.0-current master 

### Code

Provide code to help us reproduce your issues using one of the following options:

```
import tensorflow as tf
from tensorflow.keras.layers import Layer, Input
from tensorflow.keras.models import Model
import numpy as np

class UnstackLayer(Layer):
    def __init__(self, **kwargs):
        super(UnstackLayer, self).__init__(**kwargs)

    def call(self, inputs):
        unstacked = tf.unstack(inputs, axis=1)
        # only last output is used as input to the add operator
        return unstacked[-1]

input_tensor = Input(shape=(4, 16, 32))
x = UnstackLayer()(input_tensor)
output_tensor = tf.add(x, 1)

model = Model(inputs=input_tensor, outputs=output_tensor)

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

def representative_data_gen():
  for input_value in [np.random.randn(1, 4, 16, 32).astype(np.float32) for _ in range(10)]:
    yield [input_value]
converter.representative_dataset = representative_data_gen

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.target_spec.supported_types = {tf.int8}

converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()


# Save the TFLite model to a .tflite file
with open('model.tflite', 'wb') as f:
    f.write(tflite_model)
```

### Failure after conversion

Input Model:

![Screenshot from 2023-07-19 11-05-35](https://github.com/tensorflow/tensorflow/assets/112401409/dabb3863-54c3-4148-bcab-e4afd39a4ac5)

Output Model:

![Screenshot from 2023-07-19 11-28-39](https://github.com/tensorflow/tensorflow/assets/112401409/f42419d0-0365-4738-bd0c-0bf9dc24b798)


Behaviour in TF 2.11 and below is that no dequantize/quantize ops appear, which is expected.  

### Other info

The conversion started failing with version TF 2.12.0. Note that the conversion succeeds intermittently when converting the same network many times, but on average it fails. This intermittent behaviour is still present if one runs the converter on a single core and keeps the representative dataset constant. Similar issues seems to be present when converting split operators as well. 


