How Does Cross-Entropy Work With Softmax Activation Function?
As you mentioned, (output-expected)
see, this value will be less than 0 (negative as output&lt;1 and expected=1)
Now w_new = w_old - learning_rate*derivative
now based on derivative, w_new can be greater or lesser than w_old.

I guess when you are assuming with negative derivative, weights decrease is not correct.
Also even if weights decrease the output will decrease is also not correct.