About nn.Linear convert to tflite model
Hi @puyiwen 

Could you please share a toy tflite model or code to reproduce the issue? 

Thanks.

 > Hi @puyiwen
> 
> Could you please share a toy tflite model or code to reproduce the issue?
> 
> Thanks.

OK, there is my tflite model.

[qat_noquant_int8.zip](https://github.com/tensorflow/tensorflow/files/11660722/qat_noquant_int8.zip)

and there is my onnx model 
[checkpoint_best.zip](https://github.com/tensorflow/tensorflow/files/11660732/checkpoint_best.zip)
 Hi @puyiwen 

Thanks for sharing.

I have observed that Batch MatMul operation is being converted into into Split - multi FullyConnected - Pack. 

TFlite converts model ops into simple [supported operations](https://www.tensorflow.org/lite/guide/ops_compatibility) that can be run in memory constrained devices like Edge devices  and micro controllers. 

If you are facing any issue regarding quantization or want to disable quantization for any node , you can refer [Quantization Debugger](https://www.tensorflow.org/lite/performance/quantization_debugger) and do [Selective Quantization](https://www.tensorflow.org/lite/performance/quantization_debugger#selective_quantization).

Thanks. > 你好@puyiwen
> 
> 感谢分享。
> 
> 我观察到 Batch MatMul 操作正在转换为 Split - multi FullyConnected - Pack。
> 
> TFlite 将模型操作转换为简单的[支持操作](https://www.tensorflow.org/lite/guide/ops_compatibility)，这些操作可以在边缘设备和微控制器等内存受限设备中运行。
> 
> 如果您遇到有关量化的任何问题或想要禁用任何节点的量化，您可以参考[量化调试器](https://www.tensorflow.org/lite/performance/quantization_debugger)并执行[选择性量化](https://www.tensorflow.org/lite/performance/quantization_debugger#selective_quantization)。
> 
> 谢谢。

Thank you for your reply. The MatMul  operator is 'nn.Linear' in Pytorch. I am wondering why **nn.Linear** is **Batch MatMul** in tflite. How can I convert the Batch MatMul to norm MatMul? Can you help me? Hi @puyiwen 

The model can be analyzed using [Model Analyzer](https://www.tensorflow.org/lite/guide/model_analyzer) API and it can be observed that multiple MatMul operations are generated. Please find the [gist](https://colab.research.google.com/gist/pjpratik/404adf7abcaefd9e759ef10593d21167/60780.ipynb) here.

The TFlite converts model ops into  [supported operations](https://www.tensorflow.org/lite/guide/ops_compatibility)  that are natively supported by TensorFlow Lite.

Is there any challenge in using the current conversion  in your use case?

Thanks. > Hi @puyiwen
> 
> The model can be analyzed using [Model Analyzer](https://www.tensorflow.org/lite/guide/model_analyzer) API and it can be observed that multiple MatMul operations are generated. Please find the [gist](https://colab.research.google.com/gist/pjpratik/404adf7abcaefd9e759ef10593d21167/60780.ipynb) here.
> 
> The TFlite converts model ops into [supported operations](https://www.tensorflow.org/lite/guide/ops_compatibility) that are natively supported by TensorFlow Lite.
> 
> Is there any challenge in using the current conversion in your use case?
> 
> Thanks.

This kind of splitting will be very slow in reasoning on my deployment platform, and I wonder if it can be merged.
There is another problem. After my model is converted to tflite and statically quantized to in8, the model file size is larger than before it was not quantized. It is now about 5.6M. The file size of the model before quantization (I sent you earlier) It is about 3.6M. I compared the model structure before and after quantization, and tflite will automatically add a DeQuantize in front of the DIV operator, and then quantize it back after passing the DIV. I don't know if this part of the problem caused the model file to become larger after quantization. I don’t want to have this kind of operation. How should I do it?
It is my quantized tflite model.
[qat_quant_static_int8.zip](https://github.com/tensorflow/tensorflow/files/11674934/qat_quant_static_int8.zip) Hi @puyiwen 

Thanks for the information.

The converter adds `Quantize` and `DeQuantize` stubs for the ops that do not support int8 quantization, to give floating point view.

Also, can you try conversion in latest TF-Nightly and let us know your observation?

The converter adds unfolding matmul operation by default which is disabled in nightly version.

Thanks. This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.
### 1. System information

Linux Ubuntu 18.04
Tensorflow 2.8.0

### 2. Code

I have a transformer model with **pytorch**, and I use **onnx_tf** to change` .onnx` to` .pb`, then change` .pb` to` .tflite`. The` .pb ` to` .tflite` code is:
```
converter = tf.lite.TFLiteConverter.from_saved_model(path)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_data_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,tf.lite.OpsSet.TFLITE_BUILTINS_INT8,tf.lite.OpsSet.SELECT_TF_OPS]

tflite_model_quant = converter.convert()


import pathlib

tflite_models_dir = pathlib.Path('/home/puyiwen/deltar/')
tflite_models_dir.mkdir(exist_ok=True, parents=True)

# Save the quantized model:
tflite_model_quant_file = tflite_models_dir/"qat_quant_static_int8.tflite"
tflite_model_quant_file.write_bytes(tflite_model_quant)

converter = tf.lite.TFLiteConverter.from_saved_model(path)
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS]
tflite_model_noquant = converter.convert()
tflite_model_noquant_file = tflite_models_dir/"qat_noquant_int8.tflite"
tflite_model_noquant_file.write_bytes(tflite_model_noquant)
```
Then I use **Netron** to check the tfilte model `qat_noquant_int8.tflite`, and I find the **nn.Linear** (which in **ONNX** is **MatMul**) in tflite model is disassembled into three operators: **spill-fullyconnected-pack**, I dont know why and I don't want it to be dismantled. 
Another question is there is a **DIV** operator in my model, I find that when tflite is quantized, it will automatically add a **Dequantize** before the **DIV** operator. This **Dequantize** is not in my code. I don't know why tflite will add **Dequantize** by itself, and I don't want it to be added. How should I modify the conversion tflite and statically quantize the code? Thank you very much!!

The **MatMul** in onnx is 
![onnx_image](https://github.com/tensorflow/tensorflow/assets/56880072/9d67d15f-5bab-4d0d-a80d-bbd250351b76)
The **MatMu**l in tflite is:
![tflite_image](https://github.com/tensorflow/tensorflow/assets/56880072/5eed9f64-6125-454a-bea3-b643794dbe54)

