Ahmadzei's picture
update 1
57bdca5
raw
history blame contribute delete
145 Bytes
So when this training was done under fp16 mixed precision the very
last step overflowed (since under fp16 the largest number before inf is 64e3).