Keras weird loss and metrics during train I am doing some testing with tensorflow, and I bumbed into a very weird behaviour.
Here is my code
fashion_mnist = tf.keras.datasets.fashion_mnist
(train_images1, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
train_images = train_images1[:32] / 255.0
train_labels = train_labels[:32]
test_images = test_images / 255.0

batch_size = 32
epochs     = 1

train_data = tf.data.Dataset.from_tensor_slices((train_images, train_labels)).batch(batch_size)
    
adam = tf.keras.optimizers.Adam(lr=0.002, beta_1=0.9, beta_2=0.999)

input_layer   = tf.keras.layers.Input(shape=(28,28,))
flatter       = tf.keras.layers.Flatten()(input_layer)
dense1        = tf.keras.layers.Dense(128,
                                     kernel_regularizer=tf.keras.regularizers.l2(0.01), 
                                     activation='relu')(flatter)                               
dense2        = tf.keras.layers.Dense(64,
                                     kernel_regularizer=tf.keras.regularizers.l2(0.01), 
                                     activation='relu')(dense1)
output_layer = tf.keras.layers.Dense(10,
                                     activation='softmax',name='output')(dense2)
model_naive = tf.keras.models.Model(inputs=input_layer,outputs=output_layer)

model_naive.compile(optimizer=adam,
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['accuracy'])
model_naive.summary()
history = model_naive.fit(x=train_data, validation_data=train_data, epochs=epochs)
model_naive.evaluate(train_data)

I simply import fashon_mnist (only one batch) and pass it as both train and validation data, to make comparison. No dropout in the network... so I would expect to find same loss and metric but this is the output
1/1 [==============================] - 1s 815ms/step - loss: 5.4532 - accuracy: 0.0312 - val_loss: 5.0106 - val_accuracy: 0.3125

To be sure I even did a model.evaluate() and this is what I find
1/1 [==============================] - 0s 12ms/step - loss: 5.0106 - accuracy: 0.3125

exactly the same found during training.
So, provided the evaluation is correct...what are these numbers &quot;loss: 5.4532 - accuracy: 0.0312&quot; ? I am using only one batch, so I would expect no averages over batches are involved.
Please help me to understand, this is driving me crazy. Thank you!
EDIT:
With only 1 batch keras weirdly seems to print loss and score before applying gradients. The same does not happen with more than one batch where probably some strange average is performed.
Still not solved the issue, any comment is still very welcomed!
This is because the calculation of the loss and accuracy are done before the first weight update (i.e. the model with the initialized parameters). After the loss is calculated the first time the loss is used to backpropagate the error throughout the network and to update the parameters. The loss is not calculated again during training (since this would just be an added training cost). The adjusted parameters are then used to calculate the loss for the next batch (training or validation). It would therefore make more sense to compare the training statistics of the second batch with the validation statistics of the first batch.