Questions about int4 quantization
Hi @0-chan-kor 

Can you please elaborate about your feature and please specify the use cases for this feature. Also provide few samples where int4 is used in kernels.

Thanks! @pjpratik Thank you for answer.

In order to speed up various vision models (detection, landmark, segmentation, etc.) on CPU (of course, with minimal loss of accuracy), we have deployed models by applying quantization to int8.

In the meantime, we analyzed the code after learning that int4-related commits were reflected in tflite git.
As mentioned above, we chcecked that int4 type filters (weights) are supported in conv2d, depthwise-conv2d, and fully-connected kernels.
However, the tflite converter does not have a function to quantize the filter to int4, so I modified the tflite's flatbuffer and tested it.
However, it was confirmed that there was no speed improvement. (Environment : Mac M1 Pro)

these are my footsteps.

So, to summarize my questions:
1. Is there a plan to support the function of quantizing the weight to int4 in the tflite converter?
2. For the three kernels above, int4 type input and output are not yet supported. Are there any plans to support them?


 Oh, looking at the code below, isn't it actually performing int4 in the kernel?

``` c++
  if (filter->type == kTfLiteInt4) {
    tflite::tensor_utils::UnpackDenseInt4IntoInt8(
        GetTensorData<int8_t>(filter), GetTensorShape(filter).FlatSize(),
        unpacked_filter_data.get());
    filter_data = unpacked_filter_data.get();
  } else {
    filter_data = GetTensorData<int8>(filter);
  }
```

This is done simply by unpacking int4 to int8, eventually to int8? @0-chan-kor Thanks for the information.

@sachinprasadhs Could you please look at this feature request?

Thanks. Hi @pjpratik , @sachinprasadhs 
Any updates on int4?
Thanks.
Hi, all.
Since it is an inquiry rather than an issue, I will not write a template.

Looking at the kernel side code of tflite, I saw that int4 for filter is supported in several op kernels; conv2d, depthwise-conv2d, fully-connected.

Could you tell me if there are plans to support int4 quantization in the tflite converter or support int4 for each op's input as well as filter, and if so, what milestones do you have?

Thank you :)