How to train a single Artificial Neuron to predict f(x), where f(x)=x
This will not work.
because, for this problem x=99, and when finding gradients, the results will be large numbers and updating weight and bias using learning rate 0.01 will take huge steps, and so the bias and weight will cross the solutions.
using learning rate less than 0.001 will solve this problem.
I tried learning rate 0.0001 and it worked.