TFLite Interpreter fails to load fp32/ fp16 model on iPhone with CoreML or Metal Delegate in Swift
Hi @hemangchawla, I'm trying to recreate the converted model but I don't know how your params are defined, please review this colab: https://colab.sandbox.google.com/gist/pkgoogle/4ddd628342ee82a0a0eaf946a34f5ca4/62360.ipynb. If you can finish how you get the converted model and revert it back to me, that'll be the easiest. Thanks! Hi @pkgoogle, I have updated the collab. In summary, I use the ResNet50 retinanetmodel from the tf model garden. 
I convert it to tflite and load into the iOS app (have provided reference to an example iOS app from google which can be used). And  when using the coreml or tf-metal delegates, I get the error that `Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors` Hi @hemangchawla, I think you forgot to share your colab? You can save a new gist and share it. Hi @pkgoogle , please find the notebook [here](https://colab.research.google.com/gist/hemangchawla/865e06ea33b2925ed5e19ed15d206b34/62360.ipynb) Hi @hemangchawla, I was able to create the model file, but my edits to the project are running into some issues importing the MetalDelegate. Can you do me a favor and go ahead and fork https://github.com/https-deeplearning-ai/tensorflow-2-public/tree/main, make the changes to the project in question and then let me know where the edited repo/code is? This will help me best mirror your environment to reproduce this. Thanks for your help. Hi @pkgoogle,

You need to replace `pod 'TensorFlowLiteSwift'` in the podfile with either
`pod 'TensorFlowLiteSwift', '~>2.13.0, :subspecs => ['Metal', 'CoreML']` or 
`pod 'TensorFlowLiteSwift', '~> 0.0.1-nightly', :subspecs => ['Metal', 'CoreML']`, 
and then run `pod install` again.

In the`ModelDataHandler`, you need to replace
```
var options = InterpreterOptions()
options.threadCount = threadCount
do {
  // Create the `Interpreter`.
  interpreter = try Interpreter(modelPath: modelPath, options: options)
  // Allocate memory for the model's input `Tensor`s.
  try interpreter.allocateTensors()
} catch let error {
  print("Failed to create the interpreter with error: \(error.localizedDescription)")
  return nil
}
```
with 
```
var options = InterpreterOptions()
options.threadCount = threadCount
var delegate = MetalDelegate()
do {
  // Create the `Interpreter`.
  interpreter = try Interpreter(modelPath: modelPath, options: options, delegates: [delegate])
  // Allocate memory for the model's input `Tensor`s.
  try interpreter.allocateTensors()
} catch let error {
  print("Failed to create the interpreter with error: \(error.localizedDescription)")
  return nil
}
```

Of course you need to add in `ModelDataHandler` your retinanet model enum
```
/// Information about the MobileNet SSD model.
enum MobileNetSSD {
  static let modelInfo: FileInfo = (name: "detect", extension: "tflite")
  static let labelsInfo: FileInfo = (name: "labelmap", extension: "txt")
}

// ADD THIS:
/// Information about RetinaNet model. 
enum RetinaNet {
  static let modelInfo: FileInfo = (name: "retinanet_model", extension: "tflite")
  static let labelsInfo: FileInfo = (name: "labelmap", extension: "txt")
}
```

and the `retinanet_model.tflite` file must be in the appropriate location similar to the `detect.tflite` file (see `RunScripts/download_models.sh`)

Finally, you need to update the `ViewController` where you replace `MobileNetSSD.modelInfo` and `MobileNetSSD.labelsInfo` with `RetinaNet.modelInfo` and `RetinaNet.labelsInfo` respectively.  Thanks for that, I was able to replicate your issue with those steps, summarized here for convenience:

1. Create The RetinaNet model via this colab: [here](https://colab.research.google.com/gist/hemangchawla/865e06ea33b2925ed5e19ed15d206b34/62360.ipynb)
2. Get the source code for the project:
```
git clone https://github.com/https-deeplearning-ai/tensorflow-2-public.git
cd tensorflow-2-public/C2_Device-based-TF-lite/W3/ungraded?labs/ios_apps/object_detection # this is the root of the project in question
```
3. Add the retina_net model to the project:
```
# If you're at the root of the project
cp path/to/retinanet_model.tflite ObjectDetection/Model
```
4. Edit the files as stated in the above comment (I am unable to upload the modified files, but your comment is sufficiently detailed)
5. reinstall the pod
```
pod install
```
6. run the project, error in the console:
![image](https://github.com/tensorflow/tensorflow/assets/132095473/a096a275-711b-44e2-9a55-04293ce0c4f3)

@yishuangP, can you please take a look? Thanks. Hi @pkgoogle , @yishuangP could you please share if you had some time to take a look at this? Would really appreciate your support on this.  @yishuangP @pkgoogle could it be the while loop within the non maximal suppression that is causing it to have issues with converting the dynamic sizes?  Hi @hemangchawla, can you link the code in question for your hypothesis? Hi @pkgoogle, 

Please see:
https://github.com/tensorflow/models/blob/master/official/vision/modeling/factory.py#L321
in 
https://github.com/tensorflow/models/blob/master/official/vision/modeling/factory.py#L260

The nms is applied here:
https://github.com/tensorflow/models/blob/master/official/vision/modeling/retinanet_model.py#L187

The detection_generator is found here:
https://github.com/tensorflow/models/blob/master/official/vision/modeling/layers/detection_generator.py#L1114
### 1. System information

- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
- TensorFlow installation (pip package or built from source): pip
- TensorFlow library (version, if pip package or github SHA, if built from source): 2.13
- Mobile Device: iPhone 13 Pro

### 2. Code

Provide code to help us reproduce your issues using one of the following options:

I have finetuned RetinaNet detection model from the model garden. 

The saved model takes input `(1, None, None, 3)`, even though I saved it as follows:

```
from official.vision.serving import export_saved_model_lib

export_saved_model_lib.export_inference_graph(
        input_type='image_tensor',
        batch_size=1,
        input_image_size=list(args.inp_size),
        params=exp_config,
        checkpoint_path=tf.train.latest_checkpoint(args.model_dir),
        export_dir=args.export_dir)
```

where `arg.inp_size` is `[256, 256, 3]`. 

However I convert using concrete functions to tflite model that takes input (1, 256, 256, 3)

```
with open(os.path.join(saved_model_dir, "params.yaml"), "r") as stream:
    params = yaml.safe_load(stream)
h, w, c  = tuple(params['task']['model']['input_size'])[:3]
b = 1 # batch size is 1. 
print(f"Saved model size will be set to [{b, h, w, c}]")

model = tf.saved_model.load(saved_model_dir)
concrete_func = model.signatures[ tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY] 
concrete_func.inputs[0].set_shape([b, h, w, c])

converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
tflite_model = converter.convert()
```

The result of `tf.lite.experimental.Analyzer.analyze(model_content=tflite_model, gpu_compatibility=True)` can be found [here](https://drive.google.com/file/d/1z9575V_rFLvZ_IGtX9tuO8BTyj-Pqizn/view?usp=sharing).

### 3. Failure after conversion
- Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors
- Note that the model works fine on iPhone CPU. 

