Please suggest some simple functions that I can use to test my neural network implementation
Without code you could have made to many mistakes to answer your question directly.

However, I suggest two things:


Gradient checking (e.g. this explanation, but there are many more - pretty sure also in the course)
XOR problem with a 2:2:1 network with only sigmoid activations. Plot the error surface. Compare the decision surface with that of this interactive demo.


It should look similar to this:


However, please keep in mind that the problem might not be your implementation, but rather the network architecture / hyperparameters such as the number of epochs you're training or the training data.

Also, very important: I doubt that you will get good results for $y = x^2$, except if $x$ is restricted to $[-1,1]$ or something similar simple. Please keep the domain of your output layer in mind.