TFlite model has bad performance
@Thank you for posting an issue!
The model accuracy might drop due to quantization applied to the model. To improve the accuracy you try to use post-training quantization. You can apply quantization on your trained model, and retrain it. Please refer to the following references and let us know if it helps?
https://www.tensorflow.org/lite/performance/model_optimization
https://www.tensorflow.org/model_optimization/guide/quantization/training
Thank you! Thank you @sushreebarsa for the answer! 
Could you please provide me some suggestions on how to do quantization aware training for this specific architecture? [Here](https://github.com/junyongyou/triq/blob/master/src/train/train_triq.py) is the code I use for training.
Thank you in advance!  @codingzebra333 I could see there is one duplicate [ticket](https://github.com/tensorflow/tensorflow/issues/59684) raised for the same issue. Could you please follow up with the other ticket and close this one?
Thank you! Thank you for pointing that out! I did not want to close that issue in case I get another suggestion that does not involve me retraining everything. Also, could you please provide suggestions for the best approach for quantization aware training for my use case? 
Thank you!  Hi @codingzebra333 

You can wrap the `triq` model at https://github.com/junyongyou/triq/blob/4b206b593f904f3f330ca61308a8d220f9cd1de1/src/train/train_triq.py#L80
around the `tfmot.quantization.keras.quantize_model` which requires the `tensorflow_model_optimization` toolkit. 

Please refer to the [document](https://www.tensorflow.org/model_optimization/guide/quantization/training_comprehensive_guide#quantize_whole_model) on how to quantize whole model and the tips for better model accuracy.

Also, please check the QAT training [guide](https://www.tensorflow.org/model_optimization/guide/quantization/training_comprehensive_guide) and API [compatibility](https://www.tensorflow.org/model_optimization/guide/quantization/training#api_compatibility) for applying QAT.

Thanks. Hi, thank you for the suggestion. I tried it, but I got an ValueError: Quantizing a tf.keras Model inside another tf.keras Model is not supported.
[Here](https://colab.research.google.com/drive/1BIj8iFssp0mnMTPBMk52k0r3Ibb-Pt8Q?usp=sharing) is the code. 

Thank you in advance!  Hi @codingzebra333 

The QAT has few [compatability](https://www.tensorflow.org/model_optimization/guide/quantization/training#api_compatibility) limitations as mentioned earlier. 

The quantization can be applied for [tf.keras](https://www.tensorflow.org/api_docs/python/tf/keras) with only Sequential and Functional models.

 Can you please refer the Quantization Aware Training [guide](https://www.tensorflow.org/model_optimization/guide/quantization/training_comprehensive_guide) and see if it fits for your use case?

Thanks. Hey! 
I have a functional model, so it should work for my use case. 
Thank you for such a fast response! Appreciate it!  Hi @codingzebra333 

As the error suggests, using QAT tf.keras Model inside another tf.keras Model is also not supported.

In your case I can observe that the backbone and transformer model are compiled which may not be supported to quantize. In that case we may need to separate the backbone model from the main model and quantize them separately. 

Can you check the [experiments](https://www.tensorflow.org/model_optimization/guide/quantization/training_comprehensive_guide#experiment_with_quantization) with the quantization and let us know if it helps.

Thanks.
 This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59983">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59983">No</a>

Hi, I reduced .h5 model size using TFLiteConverter, but the tflite model has a bad accuracy and is not usable. [Here](https://colab.research.google.com/drive/14zMCm0XaszVrUoWd9bMvBjSXSdPvSWuw?userstoinvite=tirumaleshk%40google.com&actionButton=1#scrollTo=JWvOm1R5WyPs) is the code I used to reduce the model size. I wanted to know - is there something wrong with reducing the model size code or with the model architecture itself - that it is not possible to reduce the size for this specific model?
I also tested both - .h5 model and .tflite model on 400 images (good and bad quality), but the .tflite model evaluates good quality images the same way as bad quality images.
Here are MAD and RMSE scores:
.tflite vs .h5 (images that are in a good quality)
MAD: 1.8
RMSE: 1.9
tflite vs .h5 (same images but in a bad quality)
MAD: 0.3
RMSE: 0.4
[Here ](https://drive.google.com/drive/folders/1YcCSJgG6R4wRJ5qeU3gUoUd2296GghoL)are some examples. First MOS score in image name is .h5 result and second one is .tflite result.
Thank you in advance!