SVDF layer implementation compatible with TFLite’s SVDF operator
@VictorDominguite TensorFlow doesn't have a dedicated SVDF layer in Keras, you can achieve a similar functionality by manually defining the sequence of operations and convert them to TFLite, where the SVDF fusion might occur.
Thank you! Hi @sushreebarsa, thank you for the response! 
Actually, my main doubt here is precisely defining what is the sequence of operations that would lead to the SVDF fusion when converting to TFLite. I already tried using [Google Research's implementation](https://github.com/google-research/google-research/blob/master/kws_streaming/layers/svdf.py) and the implementation of a "low latency svdf model" I found in [this tutorial](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/speech_commands/models.py#L458) from Tensorflow's repository (for this one I had to adapt the code to use TF2 operations equivalent to the ones used in the TF1 implementation), but none of them got fused as an SVDF operator when converting to TFLite. In fact, when listing the set of operations from the TFLite models generated from Keras models with these SVDF layers, I get all separate ops, such as `'DEPTHWISE_CONV_2D', 'FULLY_CONNECTED', 'PAD', ...`.
So I was wondering what would be the correct sequence of operations that would be recognized as an SVDF op after converting to TFLite. Hi @VictorDominguite, feels like a bug or the converter doesn't properly recognize this situation, you mentioned you had to adapt the tutorial code to TF2 operations -- would you happen to have an example model creation + conversion for this situation handy that you can share on Colab? Figured you can help us save some time since you have already done the work there. Hi @pkgoogle! So, the first few lines from the `create_low_latency_svdf_model` function found in [this file](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/speech_commands/models.py#L458-L618) correspond to the SVDF operation. What I tried to do was write a layer in TF2 that performed operations equivalent to the ones from the TF1 example. One difference worth mentioning is concerning the `is_training` flag present in the TF1 implementation. It slightly alters the behavior of the model. For simplicity, I only implemented the case where `is_training = True`.
The implementation for the TF2 layer and its conversion to TFLite can be seen in [this Colab notebook](https://colab.research.google.com/drive/1hJynxjcfrplbyif20bZZyznk-Ewl0NNG?usp=sharing). I was able to replicate with your notebook, thanks! I tried with tf-nightly and it is running into a different bug. @haozha111 can you please take a look? Thanks. one more info, I tried on docker based tensorflow 2.12.0, also can't fused to TFLite's SVDF operator
Although TFLite has a built-in SVDF operator (as listed [here](https://www.tensorflow.org/mlir/tfl_ops#tflsvdf_tflsvdfop), and whose implementation can be seen [here](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/internal/reference/svdf.h)), currently, Tensorflow doesn't have a keras layer implementation for the SVDF operation.
I was wondering what would be the sequence of operations (using Tensorflow's Python library) needed so that they could be fused and recognized as an SVDF operator after converting to TFLite.