I trained my model with frozen backbone like:

```
model.get_layer('efficientnet-b0').trainable = False

```

Now, I unfreeze backbone, compile model, start training and get accuracy closed to zero. Why?
How properly fine-tune model?

Model:

```
Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
inp1 (InputLayer)               [(None, 256, 256, 3) 0                                            
__________________________________________________________________________________________________
efficientnet-b0 (Functional)    (None, None, None, 1 4049564     inp1[0][0]                       
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 1280)         0           efficientnet-b0[0][0]            
__________________________________________________________________________________________________
dropout (Dropout)               (None, 1280)         0           global_average_pooling2d[0][0]   
__________________________________________________________________________________________________
dense (Dense)                   (None, 512)          655872      dropout[0][0]                    
__________________________________________________________________________________________________
inp2 (InputLayer)               [(None,)]            0                                            
__________________________________________________________________________________________________
head/arcface (ArcMarginProduct) (None, 15587)        7980544     dense[0][0]                      
                                                                 inp2[0][0]                       
__________________________________________________________________________________________________
softmax (Softmax)               (None, 15587)        0           head/arcface[0][0]               
==================================================================================================
Total params: 12,685,980
Trainable params: 8,636,416
Non-trainable params: 4,049,564

```