Can a neural network compute $y = x^2$?
I think the answer of @ShubhamPanchal is a little bit misleading. Yes, it is true that by Cybenko's universal approximation theorem we can approximate $f(x)=x^2$ with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of $\mathbb{R}^n$, under mild assumptions on the activation function. 


  But the main problem is that the theorem has a very important
  limitation. The function needs to be defined on a compact subsets of
  $\mathbb{R}^n$ (compact subset = bounded + closed subset). But why
  is this problematic? When training the function approximator you
  will always have a finite data set. Hence, you will approximate the
  function inside a compact subset of $\mathbb{R}^n$. But we can always
  find a point $x$ for which the approximation will probably fail. That
  being said. If you only want to approximate $f(x)=x^2$ on a compact
  subset of $\mathbb{R}$ then we can answer your question with yes.
  But if you want to approximate $f(x)=x^2$ for all $x\in \mathbb{R}$
  then the answer is no (I exclude the trivial case in which you use
  a quadratic activation function).


Side remark on Taylor approximation: You always have to keep in mind that a Taylor approximation is only a local approximation. If you only want to approximate a function in a predefined region then you should be able to use Taylor series. But approximating $\sin(x)$ by the Taylor series evaluated at $x=0$ will give you horrible results for $x\to 10000$ if you don't use enough terms in your Taylor expansion.