alternativelyin ( we will define the output of neural network as the repeated composition of linear and (componentwisenonlinear functions as we shall seethis representation of the output will provide flexible class of nonlinear functions that can be easily differentiated as resultthe training of learners via gradient optimization methods involves mostly standard matrix operations that can be performed very efficiently historicallyneural networks were originally intended to mimic the workings of the human brainwith the network nodes modeling neurons and the network links modeling the axons connecting neurons for this reasonrather than using the terminology of the regression models in we prefer to use nomenclature inspired by the apparent resemblance of neural networks to structures in the human brain we notehoweverthat the attempts at building efficient machine learning algorithms by mimicking the functioning of the human brain have been as unsuccessful as the attempts at building flying aircraft by mimicking the flapping of birdswings insteadmany effective machine algorithms have been inspired by age-old mathematical ideas for function approximation one such idea is the following fundamental result (see [ for prooftheorem kolmogorov ( every continuous function [ with can be written as ( + = hi (xi = where { hi is set of univariate continuous functions that depend on gthis result tells us that any continuous high-dimensional map can be represented as the function composition of much simpler (one-dimensionalmaps the composition of the maps needed to compute the output (xfor given input are depicted in figure showing directed graph or neural network with three layersdenoted as xi hi xp hj zj aj (xhq pq zq aq figure every continuous function [ can be represented by neural network with one hidden layer ( )an input layer ( )and an output layer (