Static quantization on efficientNet
Silu, Leaky ReLu is not easy to be implemented in 8bit quantization and a lot of frameworks are not able to implement it without model degradation (256 numbers represent for the whole output range).
However, there is a trick to quantize this layer in 8-bit
1/ You can try to inference the training dataset and measure the output activation range of this layer to replicate the boundary from this function. This is usually called PostQuantize
2/ Reduce the complexity of the activation function. SiLu x*sigmoid(x) => HardSwish x*relu6(x + 3)*0.166666667. This is the idea of ReLu => ReLu6 where the output is bounded from 0-6.
I've tried to perform static quantization with a trained EfficientNetB0. Once I apply the torch.quantization.convert to transform the model from float32 to int8, an error occurs, specifically: NotImplementedError: Could not run 'aten::silu.out' with arguments from the 'QuantizedCPU' backend. I wonder if anyone has run into the same error and been able to resolve it. I have also tried with mobilenet and I get an error of the style (not the same).
Thank you very much in advance.