1. A learning machine comprising: a neural network which operates on input data with weight values to calculate output data, the neural network including a weight value storing means for storing the weight values; an error computing means which receives (i) the output data from the neural network and (ii) desired output data for calculating error values representing errors between the neural network output data and the desired output data; a gradient computing means for calculating error gradient values from at least one of the weight values and neural network output data and the error values, the gradient computing means being connected with the weight value storing means, the neural network, and the error computing means; a conjugate vector computing means for generating a present line search conjugate vector from the error gradient value and a previously calculated line search conjugate vector, the conjugate vector computing means being connected with the gradient computing means to receive the error gradient value therefrom; a line search computing unit connected with the weight value storing means, the error computing means for receiving the error values therefrom, the gradient computing means for receiving the error gradient values therefrom, and the conjugate vector computing means for receiving the present line search conjugate vector therefrom, the line search computing unit including: means for setting a line search direction based upon the error values from the error computing means, the present line search conjugate vector from the conjugate vector computing means, and the weight values at a search start point; means for determining a present search point having a minimum error based on said error values from the error computing means, said present line search conjugate vector from the conjugate vector computing means, and the weight values at the present search point, during line searches, said present search point being determined under a limiting condition that a difference between weight values at said search start point and said present search point does not exceed a predetermined upper limit; means for determining changes in weight values corresponding to said present search point and for changing the weight values stored in the weight value storing means accordingly; and means for determining as a next line search start point a sum of the weight values at said present search start point and said determined changes in the weight values.