Backpropagation in multiple output neural networks I want to understand how the backpropagation algorithm would work on a neural network with multiple outputs. 

More specifically, I have a network with 21 binary (0/1) outputs and I want to minimize the number of outputs that I get correctly; in other words, I want to minimize the hamming distance between the output vector and the desired vector.

How does the loss function work here? How do I backpropagate the error and update the weights? 
I know this might be long to explain so I'm also happy with links to some good references that I could read on the matter.
You have to pick a loss function before you can apply backpropagation or train your network.  Once you do, you can apply backpropagation.  Backpropagation doesn't tell you how to pick a loss function; that's something you have to do, based on what you're trying to achieve.

Neural networks don't have binary outputs.  Rather, they have continuous outputs.  You might want to use a logistic loss on each of the 21 outputs, and sum up those 21 losses and use that as your overall loss function.  The logistic loss is effectively a generalization of the 0-or-1 loss to the case where you have a continuous output and you want to predict either 0 or 1.