this model has lot more coefficients (also called weightsto learnthere is one between every input and every hidden unit (which make up the hidden layer)and one between every unit in the hidden layer and the output computing series of weighted sums is mathematically the same as computing just one weighted sumso to make this model truly more powerful than linear modelwe need one extra trick after computing weighted sum for each hidden unita nonlinear function is applied to the result--usually the rectifying nonlinearity (also known as rectified linear unit or reluor the tangens hyperbolicus (tanhthe result of this function is then used in the weighted sum that computes the outputy the two functions are visualized in figure - the relu cuts off values below zerowhile tanh saturates to - for low input values and + for high input values either nonlinear function allows the neural network to learn much more complicated functions than linear model couldin[ ]line np linspace(- plt plot(linenp tanh(line)label="tanh"plt plot(linenp maximum(line )label="relu"plt legend(loc="best"plt xlabel(" "plt ylabel("relu( )tanh( )" supervised learning