EfficientNet transfer learning - init with imagenet weights changes model
@google-admin can you assign someone manually to this case? It seems the bot missed it after changing the title? @maksym33 ,

Apologies for the bug in auto-assignment.I have replicated the issue and confirm difference in outputs before and after reloading the model as per attached [gist](https://colab.sandbox.google.com/gist/SuryanarayanaY/85d9b42b4b87e101cc252bb1d8e9c39d/62193_efficientnet_bug.ipynb#scrollTo=CCzU08ac5Yey). Need to check the root cause. Thanks. @SuryanarayanaY  no worries, for the root cause please see the efficient net building function, which adds additional rescaling layer for imagenet weights.
See original post for the GitHub link.
### Issue type

Bug

### Have you reproduced the bug with TensorFlow Nightly?

No

### Source

binary

### TensorFlow version

2.14

### Custom code

No

### OS platform and distribution

Linux Ubuntu 22.04

### Mobile device

_No response_

### Python version

3.10

### Bazel version

_No response_

### GCC/compiler version

_No response_

### CUDA/cuDNN version

11.8

### GPU model and memory

_No response_

### Current behavior?

My current pipeline of:
1. Train a model using EfficientNet backbone with `weights="imagenet"`
2. Transfer a trained model **weights** to the prod environment 
3. Initialize the production model with `weights=None` and load the previously learned weights
4. Compare the model's output between training and productions environment - **they are different**

Obviously, I expected the outputs to be the same. After LOTS of investigation, I found the culprit which is NOT mentioned anywhere in the documentation: [github link](https://github.com/keras-team/keras/blob/68f9af408a1734704746f7e6fa9cfede0d6879d8/keras/applications/efficientnet.py#L359).
Turns out, when the model is initialized with `weights="imagenet"` an additional rescaling layer is added. I understand the reason behind it (reproducing the Imagenet results) but the above situation proves how misleading the implementation is.

_Why didn't I just save the whole model instead of transferring only the weights?_
Well, because the model is trained with mixed precision on a GPU and transferring it this way to a production env which uses a CPU would make it unusable (nans all around).

_What I would expect ideally:_
The same model structure no matter which weights are loaded.

_What I would expect to minimize the impact of this issue:_
When the weights are loaded to production model, the mismatch between number of layers is detected and a warning is raised.

See test collab notebook with the code from below: [link](https://colab.research.google.com/drive/1M_VbMiH79elsPex08awkjDwIuvFBqOpC?usp=sharing )

### Standalone code to reproduce the issue

```shell
import tensorflow as tf
import numpy as np
from keras.applications.efficientnet import EfficientNetB3, preprocess_input

img_shape = (300,300,3)
x = np.ones((1,) + img_shape)*255

trained_model = EfficientNetB3(weights="imagenet", include_top=False, input_shape=img_shape)
trained_model.save_weights("weights.h5")

prod_model = EfficientNetB3(weights=None, include_top=False, input_shape=img_shape)
prod_model.load_weights("weights.h5")

y_trained = trained_model.predict(preprocess_input(x), verbose=0)[0,:2,:2,:2]
y_prod = prod_model.predict(preprocess_input(x), verbose=0)[0,:2,:2,:2]

print(f"Trained model output (initialized with imagenet weights):\n{y_trained}")
print(f"Production model output (initialized with 'None' weights):\n{y_prod}")
```


### Relevant log output

```shell
Trained model output (initialized with imagenet weights):
[[[-0.27523986 -0.25020567]
  [-0.26998693 -0.27772853]]

 [[-0.2728562  -0.27376127]
  [-0.24821427 -0.23993279]]]
Production model output (initialized with 'None' weights):
[[[-0.2746129   0.89038897]
  [-0.25032952  0.1611695 ]]

 [[-0.1986168   0.0976367 ]
  [-0.17414406 -0.26978934]]]
```
