Scoring metric for machine learning method
There are at least two possible interpretations of uncertainty: (a) uncertainty/confidence in the label that the network predicts; (b) uncertainty in the probability/confidence score that the network outputs for the predicted label.  (There might be others as well.)  Let me explain both of those.

Confidence scores: uncertainty in the label

Standard deep learning methods already provide a confidence score for their prediction.  They output not a single label, but a probability distribution on labels (through a final softmax layer).  Then a cross-entropy loss function is used to measure how well the predicted distribution matches the known labels, on the training set.

Once you have this, you can easily measure uncertainty.  When applied to a test instance $x$, the output will be a probability distribution on labels.  If the label $\ell$ has the highest probability (in this distribution), say probability $p$, then $p$ is a measure of the network's uncertainty/confidence: the smaller $p$ is, the more uncertain (the less confident) the network is in its prediction.

For example, suppose we have a neural network whose input is an image of a stoplight, and classifies whether it is showing a red light, i.e., the two labels are "red" or "not red".  Such a neural network (if it ends with a softmax layer) actually outputs a vector $(p_\text{red},p_\text{not red})$ where we interpret $p_\text{red}$ as the network's estimate of the probability that the light is red.  Thus, the network classifies the image as "red" if $p_\text{red} \ge p_\text{not red}$ and "not red" otherwise, and the confidence score is the probability score associated with that label.

This provides a notion of uncertainty.  For instance, suppose the network outputs the vector $(0.8,0.2)$.  Then we'll classify the image as "red", and the confidence score indicates that the network thinks there is a 80% chance that the image is red, so that is a notion of how certain it is that the image is "red".  This confidence score is also known as a "posterior probability", in a Bayesian framework, and it can be treated as measuring one kind of confidence.

Uncertainty in the confidence score

Alternatively, perhaps what we want to measure is not uncertainty in the predicted label, but uncertainty in the confidence score associated with that label.  For instance, if the output vector is $(0.8,0.2)$, then we'll classify the image as "red", with 80% confidence; perhaps what we care about is, how certain/uncertain are we, in the number "80%"?

That can be measured by the variance of this score.  We can construct 1000 different neural networks for this task.  Each one will output an output vector, and in particular, a confidence score that the image is red.  If we want our best estimate of the posterior probability that the image is red, we take the average of those 1000 scores. If we want to know the uncertainty in that estimate, we take the variance (standard deviation) of those 1000 scores.  Thus, this might let us say "we classify the image as 80% likely to be 'red', with standard deviation of 5% around the 80%".  That variance (or standard deviation) can be taken as a measure of our uncertainty in the number 80%.

This is what Nicolas Mancuso's excellent answer is explaining, in mathematical terms.  I've tried to explain it in non-mathematical terms, to help understand the idea.

Why 1000 networks?  The number 1000 is arbitrary.  A smaller number will be faster.  A larger number will provide a more precise measure of the standard deviation.

How do you train 1000 random networks?  There are several possibilities.  One is to use bagging.  Another is to train on the same training set, but use different random values when creating the network (e.g., initialize each network randomly and independently; use dropout, and use different randomization for each network's dropout, and maybe enable dropout at test time too).