Automating Transfer Learning: Gradual fine-tuning of a TensorFlow model produces: “ValueError: Unknown metric function: val_loss.” exception during the fit method of the fine-tuning stage.
Hi **@john-james-ai**,
I was able to reproduce the issue on Colab using TF v2.15, Please find the [gist](https://colab.sandbox.google.com/gist/Venkat6871/1703c6e7ac97fabcbbb14e9f40003299/62795_2-15.ipynb) here for reference.

Thank you! Hi @Venkat6871 
Thanks so much for the quick response. I saw the gist and it appears to give a different error in Tf v2.15. In either event, I'm running TF 2.8 because my computer was manufactured in the Mesozoic before AVX was invented. What are your thoughts on the next steps or recommendations?

p.s. I'm using the DenseNet201 @Venkat6871 
I think I owe you a coffee! It turns out that val_loss actually isn't an acceptable metric in the compile method; although, it can be used as a monitor metric in the EarlyStopping callback.  I've tested it with other metrics and it seems to be working fine. I'll go ahead and close this. Thanks again mate! 

j2  Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62795">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/62795">No</a>

## Current behavior?

I’m attempting to automate transfer learning fine-tuning using iterative, gradual thawing of an underlying pre-trained and frozen base network. A compiled model containing the underlying, pre-trained, and frozen base network architecture is fed into an X4Learner class. The class exposes two methods: feature_extraction, and fine_tuning. The feature extraction method fits the model for a designated number of epochs and stores the last epoch as an instance variable. The fine_tuning method operates on the ‘feature_extracted’ model and performs an iterative fine-tuning process in which each iteration thaws some number of layers in the underlying base model, then recompiles it. During fine-tuning, the fit method produces the following exception.

ValueError: Unknown metric function: val_loss. Please ensure this object is passed to the `custom_objects` argument.

A reproducible test case can be found [here](https://colab.research.google.com/drive/1B3kngb1hQr8zrnBOUoDtSuC3QKU-KeWB?usp=sharing)

I've included the code below for convenience.

### Issue type
Bug
### Have you reproduced the bug with TensorFlow Nightly?
No
### TensorFlow version
v2.8.0-0-g3f878cff5b6 2.8.0
### Custom code
Yes
### OS platform and distribution
Windows Subsystem for Linux: Ubuntu 22.04.2 LTS
### Python version
3.10..12
### Standalone code to reproduce the issue

```shell
from typing import Union
import os

import numpy as np
import pandas as pd
import tensorflow as tf
# ------------------------------------------------------------------------------------------------ #
class X4LearnerLite:
    """Performs transfer learning of a TensorFlow model containing a pre-trained base model.

    Two methods are exposed: extract_features, and fine_tune. The extract_features method trains
    the model on the given data using the designated learning rate. The fine_tune method
    thaws one or more layers in the model, then trains it on a decayed learning rate. Each
    fine tuning session decays the learning rate by a learning_rate_decay factor to mitigate
    catastrophic forgetting.

    Args:
        model (tf.keras.Model): Model containing a frozen, pre-trained base model.
        train_ds (tf.data.Dataset): TensorFlow training dataset.
        val_ds (tf.data.Dataset): TensorFlow validation dataset.
        base_model_layer (int): Index for the base model layer for thawing.
        learning_rate (float): The learning rate for feature extraction. Default = 0.0001
        metric (str): The metric used to evaluate model fit performance. Default = 'val_loss'
        loss (str): The loss function. Default = 'binary_crossentropy'.
        activation (str): Activation function. Default = 'sigmoid'.

    """

    def __init__(
        self,
        model: tf.keras.Model,
        base_model_layer: int,
        train_ds: tf.data.Dataset,
        val_ds: tf.data.Dataset,
        learning_rate: float = 0.0001,
        metric: str = "val_loss",
        loss: str = "binary_crossentropy",
        activation: str = "sigmoid",
    ) -> None:
        self._model = model
        self._base_model_layer = base_model_layer

        self._train_ds = train_ds
        self._val_ds = val_ds

        self._learning_rate = learning_rate

        self._metric = metric
        self._loss = loss
        self._activation = activation
        # Used during the thawing process to determine number of layers to thaw as proportion of
        # total number of layers in the underlying base model.
        self._n_layers = len(self._model.layers[self._base_model_layer].layers)
        self._initial_epoch = None

    # ------------------------------------------------------------------------------------------------ #
    def extract_features(self, epochs: int = 5) -> None:
        """Performs the feature extraction phase of transfer learning

        Args:
            epochs (int): Number of epochs to execute
        """

        history = self._model.fit(
            self._train_ds,
            epochs=epochs,
            validation_data=self._val_ds,
        )

        # Save the last feature extraction epoch for fine tune phase
        self._initial_epoch = history.epoch[-1]

    # ------------------------------------------------------------------------------------------------ #
    def fine_tune(
        self,
        epochs: int = 10,
        sessions: int = 10,
        learning_rate_decay_factory: float = 0.1,
        thaw_rate: Union[float, int] = 0.05,
    ) -> None:
        """Performs iterative fine tuning using gradual unfreezing of the base model.

        Args:
            epochs (int): Number of epochs per session. Default = 10
            sessions (int): Number of fine tuning sessions to execute. Default is 10
            learning_rate_decay_factor (float): Factor by which the learning rate is reduced each session.
            thaw_rate (Union[float, int]): Rate by which layers are thawed. This can be a raw
                integer or a float proportion of base model layers. Default = 0.05.
        """
        session = 0
        learning_rate = self._learning_rate
        initial_epoch = self._initial_epoch

        while session < sessions:
            session += 1

            learning_rate *= learning_rate_decay_factory

            # Thaw the top n layers of the base model according to the following
            n = max(int(self._n_layers * thaw_rate * session),1)
            self._model.layers[self._base_model_layer].trainable = True
            for layer in self._model.layers[self._base_model_layer].layers[:-n]:
                layer.trainable = False

            self._model.compile(
                loss=self._loss,
                optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate),
                metrics=[self._metric],
            )

            total_epochs = epochs + initial_epoch
            history = self._model.fit(
                self._train_ds,
                epochs=total_epochs,
                validation_data=self._val_ds,
                initial_epoch=initial_epoch,
            )

            initial_epoch = history.epochs[-1]
```


### Relevant log output
ValueError: in user code:

    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/training.py", line 1021, in train_function  *
        return step_function(self, iterator)
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/training.py", line 1010, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/training.py", line 1000, in run_step  **
        outputs = model.train_step(data)
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/training.py", line 864, in train_step
        return self.compute_metrics(x, y, y_pred, sample_weight)
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/training.py", line 957, in compute_metrics
        self.compiled_metrics.update_state(y, y_pred, sample_weight)
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/compile_utils.py", line 438, in update_state
        self.build(y_pred, y_true)
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/compile_utils.py", line 358, in build
        self._metrics = tf.__internal__.nest.map_structure_up_to(y_pred, self._get_metric_objects,
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/compile_utils.py", line 484, in _get_metric_objects
        return [self._get_metric_object(m, y_t, y_p) for m in metrics]
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/compile_utils.py", line 484, in <listcomp>
        return [self._get_metric_object(m, y_t, y_p) for m in metrics]
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/engine/compile_utils.py", line 503, in _get_metric_object
        metric_obj = metrics_mod.get(metric)
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/metrics.py", line 4262, in get
        return deserialize(str(identifier))
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/metrics.py", line 4218, in deserialize
        return deserialize_keras_object(
    File "/home/john/anaconda3/envs/bcd/lib/python3.10/site-packages/keras/utils/generic_utils.py", line 709, in deserialize_keras_object
        raise ValueError(

    ValueError: Unknown metric function: val_loss. Please ensure this object is passed to the `custom_objects` argument. See https://www.tensorflow.org/guide/keras/save_and_serialize#registering_the_custom_object for details