Can neural network represent a relation that is a derivative? Let's say I measure people hiking uphill, and it happens that their hiking speed is related to the slope so they hike slower where the hill is steeper. My input variable is the hill's elevation and my output variable is their hiking speed. I fit a neural net to this data, so that I can later predict the hiking speed given new elevation measurements. Can a neural net represent the input's first derivative and so do a good job capturing this relation? What about representing a second derivative, i.e. if hiking speed was related to the curvature of the hill?
Interesting question.

What you are asking for is if a neural network is able to represent and/or approximate functions.

The answer is: Yes, the Neural Networks can represent derivatives of functions.

In this paper, the possibility is explained.

Approximation of functions and their derivatives: A neural network implementation with applications

Other author take the same conclusion:

Multilayer feedforward networks are universal approximators

Generalization and Approximation Capabilities of Multilayer Networks

Approximation Capability of Layered Neural Networks with Sigmoid Units on Two Layers