Computability of an expression of a function rather than a function itself?
I assume that a closed form of a function is a finite expression involving function symbols from some fixed natural finite list together with some real[1] parameters. Since we know what it means to compute a real number, and since we know what it means to compute finite tuples of stuff we know how to compute with, we obtain in a direct way a notion of what it means to compute a closed form of a function.

Then in particular we can ask questions such as "given these parameters for the differential equation, can we compute a closed form of the solution".

Note however that all usual continuous functions are computable. Given a closed form of a function $f$ and some input $x \in \mathbb{R}$, we can compute $f(x)$. However, the problem "given a continuous function that has a closed form, find that closed form" is not computable. 

So asking for closed forms is asking for more, not asking for less.

[1] If you don't like computing with real numbers, go for parameters from the algebraic closure of $\mathbb{Q}[\pi,e]$ instead.