"Optimal range for loss function": Myth or Truth?
Very interesting statement you got there, i was not aware of this 'optimal loss value'.
the only way i could explain it is with the gradient computation : gradient is computed using :
$Grad = loss*lr$
So to me, changing the loss range is the same as changing the learning rate of your optimizer, it could be a good experiment to increase your lr instead of increasing the loss function range and see what happens to the training. I'm not aware of the subtilties of Adam optimizer so my explanation may be completly wrong.