feed-forward neural networks sigmoid units can be used in the output layer in conjunction with binary cross-entropy for binary classification problems the output of this unit can model bernoulli distribution over the output conditioned over figure - sigmoid function oftmax activation the softmax layer is typically used only within the output layer for multiclassification tasks in conjunction with the cross-entropy loss function refer to figure - the softmax layer normalizes outputs of the previous layer so that they sum up to one typicallythe units of the previous layer model an unnormalized score of how likely the input is to belong to particular class the softmax layer normalized this so that the output represents the probability for every class