Can someone explain me what does this loss curve says?
Follow this answer from StackOverflow, I think your problem related to the second cases where your loss gets Nan value.
Maybe you should try to use the larger datatype (for example float 16 -&gt; float 32)