```


Quantization Aware Training
Quantization-aware training(QAT) is the third method, and the one that typically results in highest accuracy of these three. With QAT, all weights and activations are “fake quantized” during both the forward and backward passes of training: that is, float values are rounded to mimic int8 values, but all computations are still done with floating point numbers. Thus, all the weight adjustments during training are made while “aware” of the fact that the model will ultimately be quantized; after quantizing, therefore, this method usually yields higher accuracy than the other two methods.

PyTorch API:

torch.quantization.prepare_qat inserts fake quantization modules to model quantization.
Mimicking the static quantization API, torch.quantization.convert actually quantizes the model once training is complete.
