Complexity of approximating a real function using queries
Not a complete answer, but hopefully a good starting point. It is very instructive to (always!) first consider the discrete analog of your question. If $X$ is some set and $f:X\to\{0,1\}$, what is the minimal number of evaluation queries needed to uniquely identify $f$? As already noted in the OP, the question only makes sense if one fixes a function class $F$ of possible candidate functions $f$ to consider.
This is a well-studied problem, where the key notion is the teaching dimension (which is the minimum size of a teaching set), see here:
https://www.cs.umd.edu/sites/default/files/scholarly_papers/NealGupta.pdf
A teaching set $S\subset X$ is such that the values of $f$ on $S$ uniquely identify $f\in F$, for a given fixed function class $F$.
Nothing is stopping you from defining an $\epsilon$-teaching dimension for real-valued functions. You can define an $\epsilon$-teaching set $S\subset X$ to be such that all $f\in F$ that agree on $S$ all lie within an $\epsilon$-tube (i.e., are all within $\epsilon$ in $\ell_\infty$ distance).
As you can see from the discussion in that paper I linked, the teaching dimension is a rather stringent notion, which motivates more &quot;interesting&quot; variants, such as the recursive teaching dimension. Here again, I encourage you to explore its natural real-valued extensions.