Understanding softmax layer in Deep Neural Network
Look at the definition of softmax: https://en.wikipedia.org/wiki/Softmax_function



The exp() gives a positive value. Since you "standardize" each value exp(e^z) in the numerator by the sum of all exp(e^z), the values sum to one in total.

Is whis what you are looking for? 

Maybe you should have a look at Hasties et al. "Elements of Statistical Learning" (Ch. 11.2 "Neural Nets"). 

Little R example:

# For standardization (denominator)
base = exp(-2) + exp(5) 

# Single exp() values are positive
exp(-2)
[1] 0.1353353
exp(5)
[1] 148.4132

# Standatdize
exp(-2)/base 
[1] 0.0009110512
exp(5)/base 
[1] 0.9990889

# Total sum equals 1
exp(-2)/base + exp(5)/base 
[1] 1