Does there exist an equivalent arithmetic circuit for each computable function?
Any computable boolean function with a fixed-length input can be computed by an arithmetic circuit.  Consider any boolean function $f:\{0,1\}^n \to \{0,1\}$.  Then there exists a multivariate polynomial $p(x_1,\dots,x_n)$ such that $f(x_1,\dots,x_n) = p(x_1,\dots,x_n)$ for all $x_1,\dots,x_n$, where arithmetic is done modulo two (i.e., over the field $\mathbb{F}_2=\{0,1\}$).  Now every multivariate polynomial can be computed by a an arithmetic circuit, so $f$ can be computed by an arithmetic circuit.

In some sense the restrictions to fixed-length inputs is unavoidable, as any circuit inherently has a fixed number of inputs and a fixed number of outputs.   So once you decide to focus on boolean functions, then the statement you saw in the crypto literature is justified: any boolean function that can be computed by a circuit, can be computed by an arithmetic circuit.  And, any computable boolean function can be computed by arithmetic circuits, where we understand "computed" to mean that there is a family of arithmetic circuits, one per input length (the non-uniform model; this is unavoidable if you want to compute using circuits, as any one circuit can only have a fixed input length).