How is PyTorch's Class BCEWithLogitsLoss exactly implemented?
According to the PyTorch documentation, the advantage of the class BCEWithLogitsLoss() is that one can use the

log-sum-exp trick for numerical stability.

If we use the class BCEWithLogitsLoss() with the parameter reduction set to None, they have a formula for that:

I now simplified the terms, and obtain after some lines of calculation:

I was curious to see whether this is the way how the Source code does it, but I couldn't find it.. The only code they have is this:
Code for BCEWithLogitsLoss

nn.BCEWithLogitsLoss is actually just cross entropy loss that comes inside a sigmoid function. It may be used in case your model's output layer is not wrapped with sigmoid. Typically used with the raw output of a single output layer neuron.
Simply put, your model's output say pred will be a raw value. In order to get probability, you will have to use torch.sigmoid(pred). (To get actual class labels, you need torch.round(torch.sigmoid(pred)).) However, you don't need to do anything like that (i.e take sigmoid) when you use nn.BCEWithLogitsLoss. Here you just have to do the following-
criterion = nn.BCEWithLogitsLoss()
loss = criterion(pred, target) # pred is just raw nn output

Hence, coming to implementation part, criterion accepts two torch tensors - one being the raw nn outputs, the other being the true class labels, then wraps the first using sigmoid - for each element in the tensor and then calculates Cross Entropy loss (-(target*log(sigmoid(pred))) for each pair and reduces it to mean.
