title: How is the categorical_crossentropy implemented in keras?
body: <p>I'm trying to apply the concept of distillation, basically to train a new smaller network to do the same as the original one but with less computation.</p>
  
  <p>I have the softmax outputs for every sample instead of the logits.</p>
  
  <p>My question is, how is the categorical cross entropy loss function implemented?
  Like it takes the maximum value of the original labels and multiply it with the corresponded predicted value in the same index, or it does the summation all over the logits (One Hot encoding) as the formula says:</p>
  
  <p><a href="https://wikimedia.org/api/rest_v1/media/math/render/svg/0cb6da032ab424eefdca0884cd4113fe578f4293" rel="noreferrer"><img src="https://wikimedia.org/api/rest_v1/media/math/render/svg/0cb6da032ab424eefdca0884cd4113fe578f4293" alt="enter image description here"></a></p>