In pytorch, which part cost most time? Loss calculation or backward+step?
In pytorch, which part cost most of time?

Loss calculation(no backward, just calculating)
backward
step
Please explain a bit if you can.


Not just in Pytorch, but in all framework, backward_step takes more time
