TFlite memory allocation thrashing with FlexDelegates
@sschaetz,
Could you please provide the complete standalone code to reproduce the issue which helps us to analyse the issue in an effective way. Thank you! This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you. This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62013">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62013">No</a>
 It should be possible to build with both optimization and symbols.
E.g.

```
bazel build --define=tflite_keep_symbols=true -c opt --copt=-g --strip=never \
     //tensorflow/lite/delegates/flex:libtensorflowlite_flex_jni.so
```
### 1. System information

- OS Platform and Distribution macOS, iOS 16.6.1 iPhone 13 Pro:
- TensorFlow installation (pip package or built from source): 2.13.0
- TensorFlow library (version, if pip package or github SHA, if built from source): tflite built from 2.13.0 tag

### 2. Code

I have a model that converts correctly and flags the following flex delegates:

```python
  tf.AddV2(tensor<1x33xcomplex<f32>>, tensor<complex<f32>>) -> (tensor<1x33xcomplex<f32>>) : {device = ""}
  tf.AddV2(tensor<complex<f32>>, tensor<complex<f32>>) -> (tensor<complex<f32>>) : {device = ""}
  tf.Complex(tensor<1x33xf32>, tensor<1x33xf32>) -> (tensor<1x33xcomplex<f32>>) : {device = ""}
  tf.Complex(tensor<f32>, tensor<f32>) -> (tensor<complex<f32>>) : {device = ""}
  tf.ConcatV2(tensor<1x1x1xcomplex<f32>>, tensor<1x1x64xcomplex<f32>>, tensor<i32>) -> (tensor<1x1x33xcomplex<f32>>) : {device = ""}
  tf.GatherV2(tensor<1x1x33xcomplex<f32>>, tensor<i32>, tensor<i32>) -> (tensor<1x33xcomplex<f32>>) : {batch_dims = 0 : i64}
  tf.Pow(tensor<complex<f32>>, tensor<complex<f32>>) -> (tensor<complex<f32>>) : {device = ""}
  tf.RealDiv(tensor<1x33xcomplex<f32>>, tensor<1x33xcomplex<f32>>) -> (tensor<1x33xcomplex<f32>>) : {device = ""}
  tf.SelectV2(tensor<1x1x1xi1>, tensor<1x1x33xcomplex<f32>>, tensor<1x1x33xcomplex<f32>>) -> (tensor<1x1x33xcomplex<f32>>) : {device = ""}
  tf.StridedSlice(tensor<1x1x33xcomplex<f32>>, tensor<3xi32>, tensor<3xi32>, tensor<3xi32>) -> (tensor<1x1x1xcomplex<f32>>) : {begin_mask = 7 : i64, ellipsis_mask = 0 : i64, end_mask = 3 : i64, new_axis_mask = 0 : i64, shrink_axis_mask = 0 : i64}
  tf.StridedSlice(tensor<1x1x33xcomplex<f32>>, tensor<3xi32>, tensor<3xi32>, tensor<3xi32>) -> (tensor<1x1x64xcomplex<f32>>) : {begin_mask = 3 : i64, ellipsis_mask = 0 : i64, end_mask = 7 : i64, new_axis_mask = 0 : i64, shrink_axis_mask = 0 : i64}
  tf.Sub(tensor<complex<f32>>, tensor<complex<f32>>) -> (tensor<complex<f32>>) : {device = ""}
  tf.Transpose(tensor<1x1x33xcomplex<f32>>, tensor<3xi32>) -> (tensor<1x1x33xcomplex<f32>>) : {device = ""}
```

### 3. Failure after conversion
When running this model on iOS, and profiling the model, I can see the application spending a lot of time (30%) in `posix_memalign` that are emitted from the flex library which leads to poor performance of the model executing (highlighted in following image):

![image](https://github.com/tensorflow/tensorflow/assets/192171/f4950106-1b7f-47cc-8025-d3fc9818e643)

Is there a way to:
- figure out which flex delegate is causing this? (If I compile the flex library with symbols, I can't seem to turn on optimization which results in an unusable library for me)
- modify the memory allocation/de-allocation strategy to re-ruse existing memory? I suspect this issue is a side-effect of tflite <-> tensorflow interop but that is only a suspicion
- set something like max allowed persistent memory to prevent de-allocation? I also have a suspicion that it might not be tied to a single delegate but by the allocator itself