example (nonlinear multi-output regressiongiven the input and an activation function rthe output ( :[ ( ) ( )]of nonlinear multioutput regression model can be computed via neural network withz where xp , ( , ) (xw where xp which is neural network with one hidden layer and output function (zz in the special case where and we collect all parameters into the vector th[ + the neural network can be interpreted as generalized linear model with [ xh([ xthfor some activation function example (multi-logit classificationsuppose thatfor classification probleman input has to be classified into one of classeslabeled we can perform the classification via neural network with one hidden layerwith nodes in particularwe have ( )softmax where is the softmax functionexp(zsoftmax exp(zk for the outputwe take ( [ ( )gc ( )] which can then be used as pre-classifier of the actual classifier of into one of the categories is then argmax gk+ (xk { , - this is equivalent to the multi-logit classifier in section notehoweverthat there we used slightly different notationwith instead of and we have reference classsee exercise in practical implementationsthe softmax function can cause numerical overand under-flow errors when either one of the exp(zk happens to be extremely large or exp(zk happens to be very small in such cases we can exploit the invariance property (exercise )softmax(zsoftmax( for any constant using this propertywe can compute softmax(zwith greater numerical stability via softmax( maxk {zk when neural networks are used for classification into classes and the number of output nodes is then the gi (xmay be viewed as nonlinear discriminant functions