TF2 keras model.fit() VS model.evaluate print different loss and AUC
The **model.evaluat**e method is being called with **train_data** as the input, which suggests that it is being evaluated on the entire training set, whereas **model.fit** is being called with a **train_dataset** that has been batched and shuffled. This difference in the input data may cause the difference in AUC scores.

The issue might be with the evaluation process also,  When using **model.evaluate()** to compute the AUC, the labels are provided as [**y_train.output_1, y_train.output_2, y_train.output_3]**, but during training, the labels are provided as **(y_train.output_1, y_train.output_2, y_train.output_3)** within the **tf.data.Dataset**.

Please check  this improved  code working: 
```
eval_dataset = tf.data.Dataset.from_tensor_slices((train_data, (y_train.output_1, y_train.output_2, y_train.output_3))).batch(batch_size)

auc = model.evaluate(eval_dataset)

# OR, to get the AUC values separately for each output:
output1_auc, output2_auc, output3_auc = model.evaluate(eval_dataset)
``` Hi @jasonxcx, 

The metrics on the training set are just the mean over all batches during **model.fit**, as the weights are changing with each batch. Using **model.evaluate** will keep the model weights fixed and compute loss/accuracy for the whole data you give in. 
Kindly refer to the [official documents](https://keras.io/getting_started/faq/#why-is-my-training-loss-much-higher-than-my-testing-loss). Thank you!  > The **model.evaluat**e method is being called with **train_data** as the input, which suggests that it is being evaluated on the entire training set, whereas **model.fit** is being called with a **train_dataset** that has been batched and shuffled. This difference in the input data may cause the difference in AUC scores.
> 
> The issue might be with the evaluation process also, When using **model.evaluate()** to compute the AUC, the labels are provided as [**y_train.output_1, y_train.output_2, y_train.output_3]**, but during training, the labels are provided as **(y_train.output_1, y_train.output_2, y_train.output_3)** within the **tf.data.Dataset**.
> 
> Please check this improved code working:
> 
> ```
> eval_dataset = tf.data.Dataset.from_tensor_slices((train_data, (y_train.output_1, y_train.output_2, y_train.output_3))).batch(batch_size)
> 
> auc = model.evaluate(eval_dataset)
> 
> # OR, to get the AUC values separately for each output:
> output1_auc, output2_auc, output3_auc = model.evaluate(eval_dataset)
> ```

Hi @pat749 
Thank you so much, but unluckily it still not work, my training result is 80+%, while my evaluation result is 60+%
 Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60177">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60177">No</a>
 neg labels and pos label are clustered, and the shuffle() seems not working
<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Support

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

2.11

### Custom Code

Yes

### OS Platform and Distribution

_No response_

### Mobile device

_No response_

### Python version

3.7.3

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

_No response_

### GPU model and memory

_No response_

### Current Behaviour?

```shell
My model generate 3 binary outputs with MMoE module, and I define a custom metric to combine these three AUC in callbacks. As I am using model.fit to train my dataset,  it prints the final training AUC in the last epoch is 0.98. While I am using model.evaluate() on the SAME training dataset, it turns out the AUC is only 0.65. Can anyone can help me with it? Btw, I do not use BatchNormalization and Dropout layers, and also the batch_size is the same for both fitting and evaluation process.

I used exactly same dataset and same batch_size, I really don’t know why it occurs?
```


### Standalone code to reproduce the issue

```shell
class MergeMetrics(tf.keras.callbacks.Callback):
    def __init__(self,**kargs):
        super(MergeMetrics,self).__init__(**kargs)

    def on_epoch_begin(self,epoch, logs={}):
        return

    def on_epoch_end(self, epoch, logs={}):
        logs['merge_auc'] = 0.7 *logs["output_1_auc"]+ 0.2*logs["output_2_auc"] + 0.1*logs["output_3_auc"]


model = Modelname(some_parameters)
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate, beta_1, beta_2),
            loss={
                "output_1_auc": ‘binary_crossentropy’
                "output_2_auc": ‘binary_crossentropy’
                "output_3_auc": ‘binary_crossentropy’
            },
            loss_weights=[1,1,1],
            metrics={
                "output_1_auc": [tf.keras.metrics.AUC(name='auc')],
                "output_2_auc": [tf.keras.metrics.AUC(name='auc')],
                "output_3_auc": [tf.keras.metrics.AUC(name='auc')]})


checkpoint = MergeMetrics()
checkpoint_filepath = os.path.join(path,'model')
model_check = tf.keras.callbacks.ModelCheckpoint(
        filepath = checkpoint_filepath,
        # filepath = path,
        monitor= "merge_auc",
        save_best_only = True,
        mode='max',
        save_weights_only = True,
        save_freq="epoch")

train_dataset = tf.data.Dataset.from_tensor_slices((train_data,                                           (y_train.output_1, y_train.output_2, y_train.output_3))).shuffle(10*batch_size).batch(batch_size)

model.fit(train_dataset, epochs=5, callbacks=[checkpoint,model_check])
model.load_weights(checkpoint_filepath) ## load weights of the best epoch
auc  = model.evaluate(train_data, [y_train.output_1, y_train.output_2, y_train.output_3], batch_size=batch_size)
```


### Relevant log output

```shell
RESULT:
Fit process : output_1_auc:0.9862,  output_2_auc: 0.9665, output_3_auc:0.5014, merge_auc: 0.9338
Evaluation process: output_1_auc: 0.7124500870704651, output_2_auc:0.6924731135368347, output_3_auc: 0.6774155497550964
```
</details>