title: Why is a TFLite model derived from a quantization aware trained model different different than from a normal model with same weights?
body: <p>I am training a Keras model that I want to deploy with TFLite in a quantized, 8-bit environment (microcontroller). To improve quantization performance, I perform quantization aware training. I then create the quantized TFLite model using my validation set as a representative dataset. Performance is evaluated using the validation set and illustrated in this image:</p>
  <p><a href="https://i.stack.imgur.com/9hz2V.png" rel="nofollow noreferrer">Error rate for various batches of 20 runs in different conditions</a></p>
  <p>If instead of simply generating the TFLite model (cyan in the figure) from the QA-trained model (red in the figure) I copy the weights from the QA-trained model to the original one and then generate the TFLite model to <a href="https://stackoverflow.com/a/66437982/15178750">work around an issue</a> (purple in the figure), this gives slightly different predictions. <strong>Why is that?</strong></p>
  <p>I understand that the TFLite models would be slightly different than the QA-trained model, since the transformation uses a post-training quantization based on the validation set. But shouldn't the quantization be the same if the structure, weights and biases of the network are the same?</p>
  <p>Sub-question: why is the TFLite model on average slightly worse than the normal Keras model? Since I am quantizing and evaluating on the validation set, if anything I would expect it to perform artificially better.</p>