Confused with backprop in pytorch with BCE loss
BCELoss ( Binary Cross Entropy Loss) is used for binary classifier, which is a neural network that have a binary output, 0 or 1. It is not used for multi-output neural network like your case. For that kind of networks, you can use MSELoss or CrossEntropyLoss as your loss for the network. 
For the calculation of BCE, it is shown on pytorch documentation.
https://pytorch.org/docs/stable/nn.html#BCELoss
For across batch, it either sum the loss or take the mean. you can set that through the reduction="" argument. 
As i said, the loss is either summed or taken the mean. Then there will be a single value so pytorch can do backdrop.


Hope I can help you