feed-forward weight matrix bias vector feed-forward neural networks in neural network with + layersthe zero or input layer ( encodes the input feature vector xand the last or output layer ( lencodes the (multivaluedoutput function (xthe remaining layers are called hidden layers each layer has number of nodessay pl nodes for layer in this notationp is the dimension of the input feature vector andfor examplepl signifies that (xis scalar output all nodes in the hidden layers ( are associated with pair of variables (za)which we gather into pl -dimensional column vectors zl and al in the so-called feed-forward networksthe variables in any layer are simple functions of the variables in the preceding layer in particularzl and al- are related via the linear relation zl wl al- bl for some weight matrix wl and bias vector bl within any hidden layer the components of the vectors zl and al are related via al sl (zl )where sl pl pl is nonlinear multivalued function all of these multivalued functions are typically of the form sl ( [ ( ) (zdim( )] ( where is an activation function common to all hidden layers the function sl pl- pl in the output layer is more general and its specification dependsfor exampleon whether the network is used for classification or for the prediction of continuous output four-layer ( network is illustrated in figure input layer output layer hidden layers bias xi ji weight , , , , , ,mk , (xfigure neural network with the layer is the input layerfollowed by two hidden layersand the output layer hidden layers may have different numbers of nodes the output of this neural network is determined by the input vector (nonlinearfunctions {sl }as well as weight matrices wl [wl, and bias vectors bl [blj for