4 Neurons to Decide 10 Digits
You could use 4 neurons to produce a 4-bit output, which then represents the digit in binary.  A digit in the range 0-9 can be expressed in binary as a 4-bit number.

However, classification accuracy might potentially be worse than the standard method of 10 output neurons with a softmax layer.