What does the "Loss" value given by Keras mean? I setup my neural net to use mean square error as shown below. To my understanding (and from reading the documentation) this means that if the correct result of a row is 0.7 and the net predicts 0.8 the contribution to the loss by this entry is (0.8 - 0.7) squared = 0.01

from keras.models import Sequential
from keras.layers import Dense

#...build up neural network layers here...

net.compile(optimizer = 'adam', loss = 'mean_squared_error')
net.fit(training_data, training_results, batch_size = 4, epochs = 100)


I get the following output.

Epoch 100/100
1190/1190 [==============================] - 0s 133us/step - loss: 0.0082

Wow the the loss is tiny, my little neural network is doing so well! However if I validate the result on my original training data

prediction = net.predict(training_data)
prediction_delta = (prediction - training_results)


Although some of the values in the prediction_delta are small overall the loss is way higher than 0.0082 with single values as high as 0.44. Note that this is for the same training data used to fit the net and is not the test data (which also shows similar results) so I would expect to get the value 0.082 back. How does Keras calculate this loss number?
In this particular case, the loss is Mean squared error
i.e. the mean of the squares of the differences of your model prediction and the real value of each sample.