Difference between sum and mean in .backward()
I know we are converting the tensor in scaler than applying backward(), but when to sum and when to mean?
some_loss_function.sum().backward()
-OR-
some_loss_function.mean().backward()


There is no canonical answer to your question. Essentially what you're asking is should I average or sum my loss, as readers we have no knowledge of your problem and what this loss function corresponds to. It all depends on your use case.
Generally though, you would average over summation because you often don't wish the loss value to scale with the dimensionality of the output. Indeed high dimensionality of your output would lead to a higher value to your loss than a summation which is meant to be constant w.r.t. the dimensions of your output tensor. If you sum your loss you will end up scaling your loss value and the gradients that are inferred from it uncontrollably.
