XLA compilation slow on new calls although no recompiling is performed
I've debugged the issue further and the origin of the slow results was due to the usage of lists. The resulting behaviour remains weird however, being the increase in computation time in each new call of the model.call. A quick and easy fix was to replace the lists by tf.constants.
    list =  [0, 0.8217, 1.03, 5.12, 5.775, 5.938, 14.7207]
becomes
    list =    tf.constant([0, 0.8217, 1.03, 5.12, 5.775, 5.938, 14.7207])


 Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59713">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/59713">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Performance

### Have you reproduced the bug with TF nightly?

No

### Source

source

### Tensorflow Version

v2.7.0-rc1-69-gc256c071bb2 2.7.0

### Custom Code

Yes

### OS Platform and Distribution

Windows 10

### Mobile device

_No response_

### Python version

3.9

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

```shell
(ON CPU) After compilation I expect speed up in speed using @tf.function(jit_compile=True). This is the case, however when used in a loop some of the iterations happen almost instantly whilst others take quite a long time. I feels like the code is recompiling, however using print statements I see that this is not the case. At first I thought there could be easy and difficult input samples to the function, however for further debugging I added an additional loop that just repeats every function evaluation multiple times. Here I saw that it is just one time a very slow one (feels like recompiling) and afterwards on the same input and exact same function it's really quick. No additional print statement is executed, hinting to the fact that the function is not compiled again. 
I was wondering if this is a known issue and if so how to fix it.

The last function below is called in a main script and makes use of a tf.keras.layers.Layer object and its call function. The call function is also given below and makes use of the jit_compiled function RK_multistep.
```


### Standalone code to reproduce the issue

```shell
@tf.function(jit_compile=True)
def RK_multistep(hybMod, x_k, u_k, dt):
    print("compiling RK")
    M = 5
    DT = dt/M
    ...

def call(self, model, x_k, u_k):
        print("----- START RK repeats -----")
        t1 = time.time()
        for k in range(3):
            t3 = time.time()
            x_kk = self.method(model, x_k, u_k, self.dt)
            t4 = time.time()
            print(f"RK iteration {k} took {t4 - t3}")
        t2 = time.time()
        print(f"All RK iterations Took {t2 - t1}")
        print("----- END RK repeats -----")
        x_kk = self.RK_multistep(model, x_k, u_k, self.dt)
    return x_kk

print("----- START model in batches -----")
t1 = time.time()
for k in range(20):
    t3 = time.time()
    Xbatch = np.expand_dims(X_train[k, :], 0)       # add batch dimension: shape (1, state_dims)
    Ubatch = np.expand_dims(Utrain[1][k, :, :], 0)  # add batch dimension: shape (1, prediction_length, control_dims)
    model_input = [Xbatch, Ubatch]
    Modelout = model(model_input)                   # here the model is called that used the call as defined above
    t4 = time.time()
    print(f"Outer layer iteration {k} took {t4 - t3}")
t2 = time.time()
print(f"Took {t2-t1}")
```


### Relevant log output

```shell
----- START model in batches -----
compiling g
compiling g_inner
compiling PI_big
compiling flow
compiling flow
----- START RK repeats -----
compiling RK
compiling f
compiling forcebalance
compiling pressure_change
RK iteration 0 took 16.773661613464355
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 16.773661613464355
----- END RK repeats -----
Outer layer iteration 0 took 17.035797357559204
----- START RK repeats -----
RK iteration 0 took 0.0
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 0.0
----- END RK repeats -----
Outer layer iteration 1 took 0.015622138977050781
----- START RK repeats -----
RK iteration 0 took 0.0
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 0.0
----- END RK repeats -----
Outer layer iteration 2 took 0.0
----- START RK repeats -----
RK iteration 0 took 0.0
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 0.0
----- END RK repeats -----
Outer layer iteration 3 took 0.0
----- START RK repeats -----
RK iteration 0 took 0.0
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 0.0
----- END RK repeats -----
Outer layer iteration 4 took 0.015624761581420898
----- START RK repeats -----
RK iteration 0 took 15.472918510437012
RK iteration 1 took 0.01561427116394043
RK iteration 2 took 0.0
All RK iterations Took 15.488532781600952
----- END RK repeats -----
Outer layer iteration 5 took 15.488532781600952
----- START RK repeats -----
RK iteration 0 took 14.715417385101318
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 14.715417385101318
----- END RK repeats -----
Outer layer iteration 6 took 14.715417385101318
----- START RK repeats -----
RK iteration 0 took 13.904020547866821
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 13.904020547866821
----- END RK repeats -----
Outer layer iteration 7 took 13.921808958053589
----- START RK repeats -----
RK iteration 0 took 0.0
RK iteration 1 took 0.0
RK iteration 2 took 0.0
All RK iterations Took 0.0
----- END RK repeats -----
Outer layer iteration 8 took 0.0
...
```
</details>