Confusion around categorical cross entropy
For binary classification, they are the same thing. Imagine $y' \in \{0,1\}$

$$ H_{y'}(y) := - \sum_{i} \sum_{c} ({y_{i, c}' \log(y_{i, c}) + (1-y_{i, c}') \log (1-y_{i, c})}) $$

$$  = -
\begin{cases}
     \sum_{i} \sum_{c} \log(y_{i, c}), &amp;\text{if}&amp;  y_{i, c}' = 1  \\
     \sum_{i} \sum_{c} \log (1-y_{i, c}), &amp;\text{if}&amp;  y_{i, c}' = 0  
\end{cases}$$


  second formula wouldn't work on a problem where the labels are not mutually exclusive (label vector can have multiple 1's)


This condition is multi-labels which is a different problem. In that case, you shouldn't even use Sigmoid or Softmax. See another answer for multi-labels loss

The second one is used for multi-class classification.

Binary Classification Loss

Cross entropy Loss