time-series prediction : loss going down, then stagnates with very high variance
I found the issue, I should have done more unit testing. Upon computing the batch loss before backpropagation, one of the dimension of the "prediction" tensor was not corresonding to the "truth" tensor. The shape match but the content is not the one that was supposed to be.
This is due to how the NLL loss is implemented in pytorch which I was not aware of ...