Loss function returns x whereas tensorflow shows validation loss as (x+0.0567)
On further investigation was able to narrow it down.

The loss value is not only calculated by the response from loss function, but also the various regularizers added in the intermediate layers.

And one of the FC layers in my model was indeed having a L2 regularization term as below:

model.add(layers.Dense(1024, activation='relu', kernel_regularizer=regularizers.l2(0.1))


If I remove the l2 regularization, the loss goes indeed goes to zero if the loss function returns a zero.