Validation loss differs on GPU vs CPU
Since one could see there is no difference in the minmal loss value, this could well be an issue of differnce in floating point precision with respect to CPU and GPU. You should try to cross check unit operations in the intermediate layers(Considering its an ANN) and look for possible discrepencies to make sure it is a floating point precision variance.