Model containing LSTM does not run after conversion using ACTIVATIONS_INT16_WEIGHTS_INT8 quantization
I reproduced the error on Ubuntu with Python 3.8. Here's a snippet of what you were trying to do (without quantization) which runs fine:
```
import tensorflow as tf

inp = tf.keras.Input([1, 1], batch_size = 1, name = "input_0")
x = tf.keras.layers.LSTM(inp.shape[2], return_sequences = True)(inp)
model_lstm = tf.keras.Model(inputs=inp, outputs=x)

# convert the model to TFLite
converter = tf.lite.TFLiteConverter.from_keras_model(model_lstm)
tflite_model = converter.convert()

# allocate tensors
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()
``` Adding onto this, we can get close to what you were trying to do before using the following script
```
import tensorflow as tf
import numpy as np

inp = tf.keras.Input([1, 1], batch_size = 1, name = "input_0")
x = tf.keras.layers.LSTM(inp.shape[2], return_sequences = True)(inp)
model_lstm = tf.keras.Model(inputs=inp, outputs=x)

# pick some representative dataset
rep_data = tf.data.Dataset.from_tensor_slices(np.float32(np.random.random_sample((10,1,1,1))))

def representative_dataset():
    for data in rep_data:
        yield [data]

# convert model to TFLite with representative dataset
converter = tf.lite.TFLiteConverter.from_keras_model(model_lstm)
converter.representative_dataset = representative_dataset
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

# allocate tensors
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()
```
But, we can see that your error occurs when trying to replace ```tflite_model = converter.convert()``` with an extra step
```
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS,
    tf.lite.OpsSet.EXPERIMENTAL_TFLITE_BUILTINS_ACTIVATIONS_INT16_WEIGHTS_INT8
]
tflite_model = converter.convert()
```
So, it seems that TF Lite is encountering an issue when trying to apply the operation set ```EXPERIMENTAL_TFLITE_BUILTINS_ACTIVATIONS_INT16_WEIGHTS_INT8``` to your LSTM model. In particular, it seems to fail to prepare the LSTM layer (```UNIDIRECTIONAL_SEQUENCE_LSTM``` operation in your error message) when this operation set is used. For now, you can try using full integer or float quantization, and I'll try to see if this is a simple fix only pertaining to this specific kind of quantization. Here are some comments from whomever implemented ```EXPERIMENTAL_TFLITE_BUILTINS_ACTIVATIONS_INT16_WEIGHTS_INT8``` in the ```OpsSet``` class.

Convert model using only TensorFlow Lite operations with quantized int8
  weights, int16 activations and int64 bias.
  Specifying this will throw an error for operations that do not yet have
  quantized implementations.
  This quantization mode may be used in models for super-resolution,
  audio signal processing or image de-noising. It improves accuracy
  significantly, but only slightly increases the model size.
  WARNING: These ops are currently experimental and have not yet been
  finalized.
  They are only compatible with CPU execution, and have not been optimized for
  production. @DerryFitz I have found a fix. Try adding ```converter.target_spec.supported_types = [tf.int8]```. For example,
```
import tensorflow as tf
import numpy as np

inp = tf.keras.Input([1, 1], batch_size = 1, name = "input_0")
x = tf.keras.layers.LSTM(inp.shape[2], return_sequences = True)(inp)
model_lstm = tf.keras.Model(inputs=inp, outputs=x)

# pick some representative dataset
rep_data = tf.data.Dataset.from_tensor_slices(np.float32(np.random.random_sample((10,1,1,1))))

def representative_dataset():
    for data in rep_data:
        yield [data]

# convert model to TFLite with representative dataset
converter = tf.lite.TFLiteConverter.from_keras_model(model_lstm)
converter.representative_dataset = representative_dataset
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.int8]
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS,
    tf.lite.OpsSet.EXPERIMENTAL_TFLITE_BUILTINS_ACTIVATIONS_INT16_WEIGHTS_INT8
]
tflite_model = converter.convert()

# allocate tensors
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()
```
Perhaps this should be implemented for you automatically, seeing as you couldn't find this documented anywhere. @alexrosen45 thanks for your efforts on this, it was nice to see your reasoning, even if I was well aware of where and how the problem was happening.  However what you propose is not a solution to the problem. I was aware of the parameter you suggested to change and did not use it for a very good reason.  In effect, by setting converter.target_spec.supported_types = [tf.int8]  as you suggest,  you are telling the converter to ignore the fact that we want 16-8 quantization and and forcing it to quantize at int8 only.  Hi @DerryFitz 

I was able to reproduce this issue. Please find the gist [here](https://colab.research.google.com/gist/pjpratik/aa003c553efb3d9446deced37ff02d3b/60884.ipynb).

The similar issues on support for LSTM on 16x8 quantization is being  tracked here [#55267](https://github.com/tensorflow/tensorflow/issues/55267), [#59626](https://github.com/tensorflow/tensorflow/issues/59626)

@pkgoogle Can you please look into this. Thank you. I was also able to reproduce using the same gist, seems like a legitimate bug.

Hi @zichuan-wei, can you please take a look? Hi @DerryFitz , 

we're wondering if you may be able to resolve your issue by using [AI-Edge-Torch](https://github.com/google-ai-edge/ai-edge-torch), you can find more information here: [googleblog](https://developers.googleblog.com/en/ai-edge-torch-high-performance-inference-of-pytorch-models-on-mobile-devices/).

I have actually created a simple script for converting an LSTM model here:
```
import torch
import torchvision
import ai_edge_torch

rnn = torch.nn.LSTM(10, 20, 2)
sample_inputs = (torch.randn(5, 3, 10),)

edge_model = ai_edge_torch.convert(rnn.eval(), sample_inputs)
edge_model.export("rnn.tflite")
```
If you want to, you can actually try visualizing the result in [model-explorer](https://github.com/google-ai-edge/model-explorer) as well.

Please try them out and let us know if this resolves your issue. If you still need further help, feel free to open a new issue at the respective repos.
###  System information

Linux OpenSuse Tumbleweed
- TensorFlow installation : pip
- TensorFlow library : Tf-nightly, occurs on earlier versions too

###  Code
Converting a model containing an LSTM to a TFlite model quantized with int 16 activations and int8 weights results in an error when trying to allocate tensors to run the model subsequently.





```
import tensorflow as tf
import numpy as np

inp = tf.keras.Input([10,20], batch_size = 1, name = "input_0")
x = tf.keras.layers.LSTM(inp.shape[2],
                             return_sequences = True)(inp)
model_lstm = tf.keras.Model(inputs=inp, outputs=x)

rep_data = tf.data.Dataset.from_tensor_slices(np.float32(np.random.random_sample((10,1,10,20))))

def representative_dataset():
        for data in rep_data:
            yield {
            "input_0": data,
            }

converter = tf.lite.TFLiteConverter.from_keras_model(model_lstm)

converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS,
#comment line below to run at int 8
tf.lite.OpsSet.EXPERIMENTAL_TFLITE_BUILTINS_ACTIVATIONS_INT16_WEIGHTS_INT8
]
converter.representative_dataset = representative_dataset

calibrated_model = converter.convert()

interpreter = tf.lite.Interpreter(model_content = calibrated_model)
interpreter.allocate_tensors()
```

###  Failure after conversion
As noted above, the interpreter fails to allocate tensors for the the converted model



### Any other info / logs
Here is the traceback of the error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[48], line 2
      1 interpreter = tf.lite.Interpreter(model_content = calibrated_model)
----> 2 interpreter.allocate_tensors()

File ~/anaconda3/envs/tfnight/lib/python3.9/site-packages/tensorflow/lite/python/interpreter.py:531, in Interpreter.allocate_tensors(self)
    529 def allocate_tensors(self):
    530   self._ensure_safe()
--> 531   return self._interpreter.AllocateTensors()

RuntimeError: tensorflow/lite/kernels/unidirectional_sequence_lstm.cc:965 output_state != nullptr was not true.Node number 1 (UNIDIRECTIONAL_SEQUENCE_LSTM) failed to prepare.Failed to apply the default TensorFlow Lite delegate indexed at 0.
