<p dir="auto">Currently loss functions does not allow weighting each instance differently. For example, in machine translation, we don't want to calculate losses for PAD labels. This can be done with cross entropy by specifying label weights, but can't be done with MSELoss (which is used to train critic/baseline model if we apply RL to MT). Manually writing a new loss function is possible, but having this feature is very convenient. The weights are better be per-instance rather than per-label.</p>
<p dir="auto">It would be useful to have losses that return a batch of losses, rather than summing/mean-ing them. This is useful e.g. for adding weights on the loss that vary for each batch element, and for masking losses on the outputs of variable-length RNNs.</p>
<p dir="auto">Most common losses can be emulated using element-wise arithmetic (or in the case of the categorical NLL, a Gather operation), and that was my standard solution in lua torch. However, having them built-in is arguably cleaner and leads to more consistent code for the end user.</p>