Segmentation Fault (Core Dumped) when convert whisper with int8 quantization
Hi @SantiagoMoreno-UdeA 

I was able to reproduce this issue in TF Nightly as well. Please find the gist [here](https://colab.research.google.com/gist/pjpratik/446755d8eff42ea5815ed19fa9257418/61695.ipynb).

A similar issue is being tracked in #59716 

Does dynamic range quantization works for your case?

Thanks. Hi @pjpratik! 

Thanks for answering 

I need all model in Int8 'cause I'm attempting to run whisper inference in a NPU and this only support int8 data type. 
So Dynamic quantization is not an option for me :/. 

Looking in advance for your answer. 

Cheers!  @SantiagoMoreno-UdeA Thanks for the information.

@pkgoogle Could you please look into this issue?

Thanks. I was able to reproduce from @pjpratik's gist.

@abattery can you please take a look Hello @abattery have you had time to take a look on this?  @SantiagoMoreno-UdeA I guess MUL op used in this model requires 16bit activation in order to preserve its accuracy. I am still not sure what is going on with TFLiteconverter 
Here is the same issue I raised so long time back and no one addresses it
 https://github.com/tensorflow/tensorflow/issues/58451 When I have analyzed, observed seg fault here ...
#0  0x00007f623573d7d3 in mlir::quant::QuantizedType::getExpressedType() const () from /usr/local/lib/python3.9/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#1  0x00007f623573e1ac in mlir::quant::QuantizedType::castFromExpressedType(mlir::Type) ()
   from /usr/local/lib/python3.9/dist-packages/tensorflow/python/_pywrap_tensorflow_internal.so @nyadla-sys It seems that so far quantize Whisper it's very tricky. Thank you for your information I'll take a look.  Related to https://github.com/tensorflow/tensorflow/issues/29829 Hi there, I am facing the same issue when trying to convert whisper into int8 for running on TPU, is there any update please? Thank you. Hi @James-Shared-Studios, No the error remains. It seems that it's very low level error.  I found that tflite versions of Whisper generate NaN values when processing the -float("inf") values that are used in one part of the transformer codebase (specifically, the logits processor that kicks in when you call generate with forced tokens).  Perhaps those NaNs make the int8 quantization crash too.  I made a crude patch here, which has worked for me to stop the NaNs happening: https://github.com/nyadla-sys/whisper.tflite/discussions/15
System information
Linux 20.04
pip Tensorflow==2.12.0
using tranformers WhisperForConditionalgeneration 

I'm trying to convert from TF to tflite and quantized to int8 Whisper, using the whisper model from tranformers WhisperForConditionalGeneration. At some point the conversion crash. 
Here is the colab for more details: 
Colab: https://colab.research.google.com/drive/1oAVoUxRFZLkS1uqqFN8HdgRVk0IWAlsN?usp=sharing

Also I attach the Error Trace from my server running in CPU and also running in GPU (TITAN RTX 24GB). 
CPU: [TraceTflite.txt](https://github.com/tensorflow/tensorflow/files/12438668/TraceTflite.txt)

GPU: [TraceTflite_GPU.txt](https://github.com/tensorflow/tensorflow/files/12452007/TraceTflite_GPU.txt)
