[TensorFlow Lite Delegates] iOS run laggy on first inference
Hi **@TimYao18** ,
You could consider pre-warming the model by running a dummy inference or setting up the interpreter without any actual input data before the actual inferences start. This could help to ensure that the initial setup costs are incurred before the critical path where low-latency is crucial.
```
// Initialize interpreter and allocate tensors without actual input data
delegates = [MetalDelegate()]
interpreter = try Interpreter(modelPath: modelPath, options: options, delegates: delegates)
try interpreter.allocateTensors()

// Run a dummy inference to perform any one-time setup
try interpreter.invoke()

// Actual inference loop
for inferenceIndex in 1...numInferences {
    do {
        // Provide input data for the inference
        try interpreter.copy(data, toInputAt: 0)

        // Run the inference
        try interpreter.invoke()

        // Get the output `Tensor` to process the inference results.
        outputTensor = try interpreter.output(at: 0)
        
        // Process the output as needed

    } catch let error {
        os_log(
            "Failed to invoke the interpreter with error: %s", type: .error,
            error.localizedDescription)
        return
    }
}
```
This modification involves initializing the interpreter and allocating tensors without any actual input data before the inference loop. Then, a dummy inference is performed to handle any one-time setup. After that, the actual inferences are run in the loop. This approach can help ensure a more consistent latency across all inferences.

Thank you! Is this behavior considered as a system behavior on iOS?
I want to make sure the issue doesn't be with the delegate. 
If I just want to inference single time, ex image classification, I think it will get a little bit strange that I have to do a pre-warming inference. Hi @TimYao18, most complex systems have some sort of initial setup costs, In this case it mostly looks GPU specific ... is there a use case where this is critical? I just found this phenomenon during testing, for example, performing inference again after a minute also becomes slightly slow. But since it's considered as initial setup costs, I can only take it as a system limitation. Thank you. Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62738">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62738">No</a>

### Issue type

Performance

### Have you reproduced the bug with TensorFlow Nightly?

Yes

### Source

source

### TensorFlow version

'TensorFlowLiteSwift', '~> 0.0.1-nightly'

### Custom code

Yes

### OS platform and distribution

_No response_

### Mobile device

iPhone 15 Pro MAX

### Python version

_No response_

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current behavior?

The first inference should have nearly the same latency at other inferences.

### Standalone code to reproduce the issue

```shell
https://github.com/isl-org/MiDaS/tree/master/mobile/ios

delegates = [MetalDelegate()]
interpreter = try Interpreter(modelPath: modelPath, options: options, delegates: delegates)
try interpreter.allocateTensors()
inputTensor = try interpreter.input(at: 0)
outputTensor = try interpreter.output(at: 0)

do {
  try interpreter.copy(data, toInputAt: 0)
  try interpreter.invoke()

  // Get the output `Tensor` to process the inference results.
  outputTensor = try interpreter.output(at: 0)
} catch let error {
  os_log(
    "Failed to invoke the interpreter with error: %s", type: .error,
    error.localizedDescription)
  return
}
```


### Relevant log output

```shell
inference index,CPU,GPU,ANE,
1,83.08,123.78,20.00,
2,66.86,35.96,28.76,
3,67.70,19.67,24.16,
4,78.89,19.11,24.29,
5,69.42,19.15,24.85,
6,69.58,19.35,24.16,
7,68.47,19.29,26.29,
8,69.42,19.86,24.36,
9,69.24,19.49,24.91,
10,73.53,20.40,24.03,
11,70.91,19.29,24.62,
12,70.52,18.94,24.71,
13,71.66,19.35,24.30,
14,70.40,19.15,24.09,
15,71.73,19.37,23.74,
16,72.06,20.41,24.79,
17,70.98,19.28,23.81,
18,71.97,19.52,25.02,
19,72.19,20.14,23.95,
20,71.55,19.28,24.05,
21,73.17,19.00,24.26,
22,73.02,19.47,24.35,
23,71.86,20.24,23.90,
24,73.81,20.58,24.00,
25,72.03,19.51,24.55,
26,72.37,20.75,24.30,
27,73.91,19.23,23.61,
28,72.73,19.16,23.46,
29,72.86,19.36,24.23,
30,73.44,19.52,24.02,
31,72.76,20.09,24.99,
32,73.49,19.51,24.24,
33,76.49,20.70,23.92,
34,73.78,19.53,23.98,
35,73.45,19.01,23.92,
36,74.01,19.53,24.03,
37,74.03,20.48,23.95,
38,73.22,19.78,24.13,
39,74.21,20.78,23.45,
40,74.64,19.48,24.41,
41,73.77,19.36,23.42,
42,75.36,19.85,23.76,
43,75.76,20.30,23.87,
44,74.97,19.84,24.27,
45,74.44,19.83,23.75,
46,76.28,20.96,24.07,
47,76.44,19.36,23.70,
48,75.56,19.42,24.21,
49,74.88,20.82,24.74,
50,76.20,20.10,24.32,
51,76.29,19.62,23.08,
52,75.45,20.93,23.53,
53,75.03,19.33,23.05,
54,76.22,19.32,23.25,
55,76.57,19.69,23.67,
56,76.05,20.14,24.51,
57,75.14,19.58,23.07,
58,76.09,20.45,23.58,
59,76.01,19.37,23.57,
60,76.39,19.26,24.53,
61,76.18,19.21,23.38,
62,75.79,19.60,23.24,
63,76.43,19.75,23.20,
64,77.32,19.63,23.84,
65,77.05,19.45,22.99,
66,76.45,19.90,23.32,
67,76.25,20.06,23.38,
68,76.99,20.11,23.55,
69,77.57,20.86,23.28,
70,77.42,19.61,23.37,
71,76.84,19.92,24.12,
72,76.49,21.05,23.25,
73,76.61,20.25,23.84,
74,78.08,20.95,23.44,
75,77.66,19.91,23.47,
76,76.42,19.52,23.76,
77,77.00,19.92,23.38,
78,77.56,19.62,23.69,
79,78.40,20.31,23.65,
80,78.56,19.67,23.30,
81,77.59,19.21,24.01,
82,77.83,20.21,24.67,
83,77.07,19.80,21.67,
84,77.55,20.08,28.61,
85,77.73,19.68,31.18,
86,78.05,19.10,31.83,
87,78.54,19.70,30.31,
88,77.76,20.37,29.35,
89,77.24,19.58,27.23,
90,77.72,19.92,28.56,
91,77.66,19.38,24.71,
92,78.45,20.20,24.32,
93,79.30,20.01,25.32,
94,78.52,19.99,24.10,
95,76.68,19.76,23.12,
96,77.93,19.42,22.87,
97,77.74,20.54,22.79,
98,77.39,19.82,22.46,
99,78.75,20.26,21.99,
100,79.28,19.93,22.60,
101,78.99,19.94,20.58,
102,79.06,20.11,21.44,
103,78.65,20.09,21.25,
104,78.57,19.77,21.29,
105,77.44,19.84,20.99
```
