Backpropagation for all zero label vectors in softmax/crossentropy
@sachinprasadhs,
I was able to reproduce the issue on tensorflow v2.12 and nightly. Kindly find the gist of it [here](https://colab.research.google.com/gist/tilakrayal/79d4e1836b23cb219236e63d8ab3443f/untitled1071.ipynb). @Janushki , Thanks for reporting the issue. 
I was debugging this from my side as well.
Meanwhile, Could you please provide some more details on the mathematical computation of both of the approaches. Initial observation: 
I was able to compare both the results outside gradient scope for forward computation and the results were identical.
The issue seems to be happening in the gradient calculation. 
Below is the code I tried to compare both results with a seed and detailed gist can be found [here](https://gist.github.com/sachinprasadhs/a792cadef1ef4f217be127091c7953e6).
```
Step1 
import tensorflow as tf
import keras
tf.keras.utils.set_random_seed(42)
x = tf.constant([[0.1,0.90]])
y = tf.constant([[0.0,.0]])
z = tf.keras.Sequential(
         [keras.layers.Dense(2,activation = "linear", use_bias = False),
          keras.layers.Softmax()]
      )(x)
print(z)
z = tf.losses.categorical_crossentropy(y, z)
print(z)
```

```
tf.Tensor([[0.22173299 0.778267  ]], shape=(1, 2), dtype=float32)
tf.Tensor([-0.], shape=(1,), dtype=float32)
```

```
Step2:
import tensorflow as tf
import keras
tf.keras.utils.set_random_seed(42)
x = tf.Variable([[0.1,0.90]])
y = tf.Variable([[0.0,.0]])
z = keras.Sequential(
         keras.layers.Dense(2, activation = "softmax", use_bias = False)
      )(x)
print(z)
z = tf.losses.categorical_crossentropy(y, z)
print(z)
```

```
tf.Tensor([[0.22173299 0.778267  ]], shape=(1, 2), dtype=float32)
tf.Tensor([0.], shape=(1,), dtype=float32)
```
 > @Janushki , Thanks for reporting the issue. I was debugging this from my side as well. Meanwhile, Could you please provide some more details on the mathematical computation of both of the approaches.

Sorry for the late answer, I didnt have time last week. And thanks alot for your effort!

The Categorical Crossentropie Loss is defined as:
$$-\sum_i y_i * \log(x_i)$$

if we add the information that $y_i = 0$ this leads to
$$-\sum_i y_i * \log(x_i)=-\sum_i 0 * \log(x_i)=-\sum_i 0=0$$
as the loss is constantly zero, the partial derivate for $x_i$ is zero aswell. Actually this should be independent of the pervious activation function.
 Hi @Janushki ,

Thank you for the report. There was indeed a discrepancy in the results depending on the approach. This is now fixed in `tf-nightly`. https://github.com/keras-team/keras/commit/304bb3d9ab137dd26263381977890f00aac62e75

Specifically, this fixes the `Softmax` layer. After this change, the `Softmax` layer produces the same result as the `softmax` activation.

Thanks,
Fabien Are you satisfied with the resolution of your issue?
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=Yes&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60314">Yes</a>
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfaP12TRhd9xSxjXZjcZFNXPGk4kc1-qMdv3gc6bEP90vY1ew/viewform?entry.85265664=No&entry.2137816233=https://github.com/tensorflow/tensorflow/issues/60314">No</a>

<details><summary>Click to expand!</summary> 
 
 ### Issue Type

Bug

### Have you reproduced the bug with TF nightly?

Yes

### Source

source

### Tensorflow Version

2.13.0-dev20230413

### Custom Code

Yes

### OS Platform and Distribution

Ubuntu 22.04.1 LTS

### Mobile device

_No response_

### Python version

Python 3.10.10

### Bazel version

_No response_

### GCC/Compiler version

_No response_

### CUDA/cuDNN version

no GPU Used

### GPU model and memory

no GPU Used

### Current Behaviour?

The First approach to building a Softmax layer yields a non zero gradient in backpropagation for an all zero target vector. The second approach yields a zero gradient. I would expect the behaviour of the latter, as is mathematically correct. 
I am a bit unsure whether this is intentended behaviour or not. If it is intended, where does the difference come from? 

### Standalone code to reproduce the issue

```shell
import tensorflow as tf
import keras

x = tf.constant([[0.1,0.90]])
y = tf.constant([[0.0,.0]])

# First approach 
with tf.GradientTape() as g:
  g.watch(x)
  z = keras.Sequential(
         keras.layers.Dense(2, activation = "softmax", use_bias = False)
      )(x)
  z = tf.losses.categorical_crossentropy(y, z)
dz_dx = g.gradient(z, x)
print(dz_dx)



x = tf.constant([[0.1,0.90]])
y = tf.constant([[0.0,.0]])

# Second approach 
with tf.GradientTape() as g:
  g.watch(x)
  z = keras.Sequential(
         [keras.layers.Dense(2, activation = "linear", use_bias = False),   
          keras.layers.Softmax(axis = -1)]
      )(x)
  z = tf.losses.categorical_crossentropy(y, z)
dz_dx = g.gradient(z, x)
print(dz_dx)
```


### Relevant log output

_No response_</details>