Can a neural network compute $y = x^2$? In spirit of the famous Tensorflow Fizz Buzz joke and XOr problem I started to think, if it's possible to design a neural network that implements $y = x^2$ function?

Given some representation of a number (e.g. as a vector in binary form, so that number 5 is represented as [1,0,1,0,0,0,0,...]), the neural network should learn to return its square - 25 in this case.

If I could implement $y=x^2$, I could probably implement $y=x^3$ and generally any polynomial of x, and then with Taylor series I could approximate $y=\sin(x)$, which would solve the Fizz Buzz problem - a neural network that can find remainder of the division.

Clearly, just the linear part of NNs won't be able to perform this task, so if we could do the multiplication, it would be happening thanks to activation function.

Can you suggest any ideas or reading on subject?
I think the answer of @ShubhamPanchal is a little bit misleading. Yes, it is true that by Cybenko's universal approximation theorem we can approximate $f(x)=x^2$ with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of $\mathbb{R}^n$, under mild assumptions on the activation function. 


  But the main problem is that the theorem has a very important
  limitation. The function needs to be defined on a compact subsets of
  $\mathbb{R}^n$ (compact subset = bounded + closed subset). But why
  is this problematic? When training the function approximator you
  will always have a finite data set. Hence, you will approximate the
  function inside a compact subset of $\mathbb{R}^n$. But we can always
  find a point $x$ for which the approximation will probably fail. That
  being said. If you only want to approximate $f(x)=x^2$ on a compact
  subset of $\mathbb{R}$ then we can answer your question with yes.
  But if you want to approximate $f(x)=x^2$ for all $x\in \mathbb{R}$
  then the answer is no (I exclude the trivial case in which you use
  a quadratic activation function).


Side remark on Taylor approximation: You always have to keep in mind that a Taylor approximation is only a local approximation. If you only want to approximate a function in a predefined region then you should be able to use Taylor series. But approximating $\sin(x)$ by the Taylor series evaluated at $x=0$ will give you horrible results for $x\to 10000$ if you don't use enough terms in your Taylor expansion.