Is it possible to control asymptotic behaviour of neural network models?
For standard NNs, their extrapolation behavior an important aspect for financial applications cannot be controlled due to complex functional forms typically involved.
Neural Networks with Asymptotics Control discuss how they overcome this significant limitation and develop a new type of neural networks that incorporate large-value asymptotics, when known, allowing explicit control over extrapolation. This new type of asymptotics-controlled Neural network is based on two novel technical constructs:

A Multi-Dimensional Spline Interpolator with Prescribed Asymptotic Behavior
and A Custom NN layer that guarantees zero asymptotics in chosen directions.

Let $f(x)$  be the Function that we want to approximate, while preserving its asymptotics. (It goes without saying that the asymptotics needs to be known.) As the first step, we find a control variate function $S(x)$ that has the same asymptotics as $f(x)$.

Multi-Dimensional Inputs - the most interesting case - these asymptotics could either be known in all directions, or only in some; naturally, as much information about the asymptotics as is available should be incorporated into the control variate function $S(x)$.

For step one, they show how to construct a universal control variate, a multi-dimensional spline $S(x)$ that has the same asymptotics as $f(x)$; and that can be used in all situations. In specific applications, of course, if more is known about the function $f$; a better choice of the control variate could be available.

For step two, we design a custom NN layer that guarantees zero asymptotics in all directions, with fine control over the regions where the NN interpolation is used and where the asymptotics kick in. they approximate the residual function $R(x) = S(x) - f(x)$ with a special NN that has vanishing (zero) asymptotics in all, or some, directions.




One-Dimensional Spline with Asymptotics Control:
The paper discussed that Instead of using the natural boundary conditions, i.e. setting second derivatives
to zero at the boundaries, they have fixed the first derivatives at boundary points to
arbitrary values. $\color{blue}{\text{Clearly, this paves the way to control asymptotics}}$.
Suppose that we know the behavior of the original, calculation-heavy function in its tails, i.e. $f(x) \simeq f_{−}(x)$ for $x&lt;h_0$ and $f(x) \simeq f_{+}(x)$  for $x&gt;h_{N+1}$
for large negative $h_0$ and large positive $h_{N+1}$. First, we
expand the set of spline nodes to include points $h_0$ and $h_{N+1}$ and make sure
the spline passes through them, $S(h_0) = f_{−}(h_0)$ and $S(h_{N+1}) = f_{+}(h_{N+1})$. To
finish, they have specifies first order derivatives at these new boundary points.
$$S^{\prime}_0(h_0) = f^{\prime}_{−}(h_0), S(h_{N+1}) = f^{\prime}_{+}(h_{N+1})$$
This fully specifies the approximating function $\tilde{S}(x)$,
$$\tilde{S}(x)=\left\{\begin{array}{ll}f_{-}(x), &amp; x \leq h_{0} \\ S(x), &amp; h_{0}&lt;x&lt;h_{N+1} \\ f_{+}(x), &amp; h_{N+1} \leq x\end{array}\right.$$
One-Dimensional Spline with Asymptotics Control: Continuous Second Derivatives
where they have discussed how they have constructed a control variate function based on the spline like above, but with the continuous second derivative at the endpoints. In order to do that, they have picked a point between $h_0$ and $h_1$, say $h_{\frac{1}{2}}$ , and find, analytically, the value $S(h_{\frac{1}{2}})$ such that the second derivative at $h_{0}, S^{\prime \prime}\left(h_{0}\right)= f_{-}^{\prime \prime}\left(h_{0}\right)$ is matched.

Asymptotic Behaviour of Gradient Learning Algorithms in Neural Network Models for the Identification of Nonlinear Systems. This paper deals with studying the asymptotical properties of multilayer neural networks models used for the adaptive identification of wide class of nonlinearly parameterized systems in stochastic environment.


A study of the asymptotic behavior of neural networks. where they have studied of neural networks modeled as a set of nonlinear differential equations of the form $TX+X=Wf(X)+b$, where $X$ is the neural membrane potential vector, $W$ is the network connectivity matrix, and $F(X)$ is the nonlinearity (an essentially sigmoid function). Topologies of neural networks that exhibit asymptotic behavior are established as behavior depends solely on the topology of the network. Moreover, the connectivity $W$ need not be symmetric. The simulated behavior of typical neural networks is presented

Refrence
Neural Networks with Asymptotics Control
Asymptotic Behaviour of Gradient Learning Algorithms in Neural Network Models for the Identification of Nonlinear Systems
A study of the asymptotic behavior of neural networks