How can one measure the time dependency of an RNN?
I am not aware of any standard or widely-used metric for this. I think what metric would be appropriate would depend on what you want to use it for.
The issues with RNN is &quot;forgetting&quot;.  If you feed a long sequence of inputs $x=(x_1,\dots,x_n)$ into a RNN, where $n$ is too large, the problem is that often the final decision is determined by the last few values ($\ldots,x_{n-1},x_n$) and the earliest values ($x_1,x_2,\ldots$) have been &quot;forgotten&quot; and do not affect the final decision.  This is undesirable in many settings.
Your metric would be one reasonable way to get a feeling for this.  Another reasonable way might be to to feed in an input $x=(x_1,x_2,\dots,x_n)$, then change just $x_1$ to get a new input $x'=(x'_1,x_2,\dots,x_n)$, feed in $x'$, and compare the outputs of the RNN on $x$ vs $x'$; and repeat for many training samples or test samples $x$.