Learning from derivative data
If your function is $f:\mathbb{R}\to\mathbb{R}$, you can "learn" $f'$ as a standard regression problem (linear, polynomial, etc.) and then recover $f$ up to an additive constant by integrating $f'$. Obviously, you will only able to recover $f$ up to an additive constant from the derivative.

Update: As for error estimates, here's a simple one. Suppose you've managed to recover $f'$ up to $\epsilon$-accuracy in $L_\infty$. Suppose further that you know that $f(0)=0$ and you only care about $f$ on $[0,T]$. Then your estimate of $f$ (obtained by integrating $f'$) is accurate up to $T\epsilon$ in $L_\infty$.