Proving Monotonicity of Softmax Layer
Let's remove the $L$ superscripts. The derivative with respect to $z_L$ is
$$
\frac{\partial a_j}{\partial z_j} =
\frac{e^{z_j} \sum_k e^{z_k} - e^{z_j} e^{z_j}}{\left(\sum_k e^{z_k}\right)^2} =
\frac{e^{z_j} \sum_{k \neq j} e^{z_k}}{\left(\sum_k e^{z_k}\right)^2} &gt; 0.
$$