output unit and no hidden units, and in which each neuron uses a linear output (unlike most work on neural networks, in which mapping from inputs to outputs is non-linear) that is the weighted sum of its input. Initially, before training, the weights will be set randomly. Then the neuron learns from training examples, which in this case consist of a set of tuples where and are the inputs to the network and is the correct output (the output the network should produce given those inputs, when it has been trained). The initial network, given and , will compute an output that likely differs from (given random weights). A loss function is used for measuring the discrepancy between the target output and the computed output . For regression analysis problems the squared error can be used as a loss function, for classification the categorical crossentropy can be used. As an example consider a regression problem using the square error as a loss: where is the discrepancy or error. Consider the network on a single training case: . Thus, the input and are 1 and 1 respectively and the correct output, is 0. Now if the relation is plotted between the network's output on the horizontal axis and the error on the vertical axis, the result is a parabola. The minimum of the parabola corresponds to the output which minimizes the error . For a single training case, the minimum also touches the horizontal axis, which means the error will be zero and the network can produce an output that exactly matches the target output . Therefore, the problem of mapping inputs to outputs can be reduced to an optimization problem of finding a function that will produce the minimal error. However, the output of a neuron depends on the weighted sum of all its inputs: where and are the weights on the connection from the input units to the output unit. Therefore, the error also depends on the incoming weights to the neuron, which is ultimately what needs to be changed in the network to enable learning. In this example, upon injecting the training data , the loss function becomes Then, the loss function takes the form of a parabolic cylinder with its base directed along . Since all sets of weights that satisfy minimize the loss function, in this case additional constraints are required to converge to a unique