Why doesn't neural networks use the concept of degree of freedom?
Actually neural networks do take this information into account.

When we do a hard classification with NN (i.e. only one class for the output), we (almost always) use a softmax output layer, then taking the max of the softmax layer to get the class and its probability.

Then why do we still have 10 outputs before the softmax and why 10 after? Well, you need to have the 10 inputs to the softmax to indicate the weight the network gives to each class, as this is still not scaled back to a probability. Then to ease the usage of the output layer, we want to keep having 10 outputs to be able to use the values without reconstructing the missing value (which would need to be computed anyway due to the way softmax works).