Prediction of random variables by excursion metric projections

Abstract: We use the concept of excursions for the prediction of random variables without any moment existence assumptions. To do so, an excursion metric on the space of random variables is defined which appears to be a kind of a weighted $L^1$-distance. Using equivalent forms of this metric and the specific choice of excursion levels, we formulate the prediction problem as a minimization of a certain target functional which involves the excursion metric. Existence of the solution and weak consistency of the predictor are discussed. An application to the extrapolation of stationary heavy-tailed random functions illustrates the use of the aforementioned theory. Numerical experiments with the prediction of Gaussian, $\alpha$-stable and further heavy--tailed time series round up the paper.

\section{Introduction} \label{sectIntro}

Let $Y:\Omega\to\mathbb R$ be a square integrable random variable defined on a probability space $(\Omega,{\cal F}, \mathbb{P})$, and let ${\cal G}\subset {\cal F}$ be a sub--$\sigma$--algebra generated by a family of random variables $\{Z_j\}$  which are observable.
The classical $L^2$--theory of prediction of random variables states that the best unbiased predictor of  $Y$ with respect to  ${\cal G}$ is given by the conditional expectation $\mathbf {E}(Y | {\cal G}) $ which
is an orthogonal $L^2$--projection of $Y$ onto the space of ${\cal G}$--measurable square integrable random variables. But as far as $Y$ has no finite moments, no unified widely accepted prediction theory exists, to the best of our knowledge. Our paper is an attempt to create such theory which also applies to the finite variance case. Its main idea is the following.
Let $u\in \mathbb R$ be an excursion level chosen according to a finite measure $m(\cdot)$ on $\mathbb R$. For any two random variables $Y_1, Y_2:\Omega\to\mathbb R$  introduce the quantity
$$
E_m(Y_1,Y_2):=\mathbf {E} \int_\mathbb R \mathbbm{1}( \{ Y_1>u  \} \bigtriangleup   \{ Y_2>u  \} ) \, m(du)=\int_\mathbb R \mathbb{P}( \{ Y_1>u  \} \bigtriangleup   \{ Y_2>u  \} ) \, m(du),
$$
which is (by Fubini's theorem) a  $m$-weighted average probability of symmetric difference of excursions of $Y_1$ and $Y_2$ over $u\in\mathbb R$. Then, we say that $Z$ is a prediction of a random variable $Y$ onto the $\sigma$--algebra ${\cal G}$ introduced above if
$$
Z=\mbox{argmin}_{Y_0}  E_m(Y,Y_0),
$$
whenever this minimum (taken over all ${\cal G}$-measurable random variables $Y_0$) exists and is unique. Sometimes it is also plausible to add more constraints to the geometry of our projection space saying that, additionally to ${\cal G}$-measurability, $Y_0\stackrel{d}{=} Y$ (here, $\stackrel{d}{=}$ means the equality in law) or that $Y_0$ is a linear combination of  $\{Z_j\}$. Apparently, the above solution $Z$, its existence and uniqueness may heavily depend on the choice of measure $m(\cdot)$. A natural candidate for this would be the distribution of $Y$ as we explain it later.
As it is shown in Theorem \ref{thm:metric}, $E_m$ is a metric on the space of random variables whenever the distribution function of $m(\cdot)$ is strictly increasing.

The intuition behind the use of the new metric is the following. Assume that a stationary heavy--tailed time series $\{Y_t,t\in\mathbb R\}$ is observed at locations $t_1,\ldots, t_n$ in a compact window $W\subset \mathbb R$. As proposed in \cite{Dasetal22}, the linear predictor
$\widehat{Y}_t=\sum\limits_{j=1}^n \lambda_j Y_{t_j}$, $t\not\in \{ t_1,\ldots, t_n \}$, is a minimizer of the functional
$$
 \int_\mathbb R \mathbf {E} \left[v_{1}\left(A_{Y}(u) \Delta A_{\widehat{Y}}(u)\right)\right] \, m(du)=\int\limits_W E_m(Y_t,\widehat{Y}_t) \, dt$$
with respect to the choice of weights $\lambda_1,\ldots,\lambda_n$ subject to the constraint $\widehat{Y}_t\stackrel{d}{=} Y_t$. The above equality holds by Fubini's theorem, whereas the left hand side term is the mean length of the symmetric difference of excursion sets
$A_{Y}(u) :=\{ t\in W: Y_t>u  \}$ and $A_{\widehat{Y}}(u):=\{ t\in W: \widehat Y_t>u  \}$ averaged over the levels $u\in\mathbb R$ picked up according to the measure $m(\cdot)$. The term $\mathbf {E} \left[v_{1}\left(A_{Y}(u) \Delta A_{\widehat{Y}}(u)\right)\right] $ is called {\it error-in-measure} and quantifies the prediction error measured by the symmetric difference of excursions. Here and in what follows,  $v_1(\cdot)$ is the Lebesgue measure on $\mathbb R$. In view of the said above, $E_m$ will be named the { \it excursion metric}.

The paper is organised as follows: in Section \ref{sectExcursMetric} the properties of $E_m$ are studied. It will be shown that $E_m$ coincides with the so--called {\it separation (pseudo) metric }\cite{Taylor84} whenever $m$ is a probability measure.
The maximal value attained by $E_m$ with respect to the choice of measure $m$ as well as implications of the choice $m=\mathbb{P}_Y$ are given in Section \ref{sectOptChoice}.
Restricted to the space of random variables $Y$ with the same absolutely continuous distribution $F$,  the metric $E_{F}$ turns to be distribution--free depending only on bivariate copulas. We call this metric (in analogy to Gini coefficient from econometrics \cite{Gast72,Yitz13}) a {\it Gini metric}. It properties are investigated within the same section.
The excursion metric with $ m=\mathbb{P}_Y$ is applied to the prediction of random variable $Y$  in Section \ref{sectPredict}. There, we give three possible forms of minimization problems
leading to such forecasting.   A special case of extrapolating heavy-tailed time series is considered in Section \ref{sectExtrapol}. Numerical examples predicting Gaussian and $\alpha$--stable processes  follow in Section \ref{sect:Num}.

\section{Excursion metric and its properties} \label{sectExcursMetric}

We introduce the {\it excursion (pseudo)metric} mentioned in Section \ref{sectIntro} in a slightly different (but equivalent) form:
\begin{definition}
\label{Em:def}
Let $m$ be a finite non-negative measure on $\mathbb R,$ and $Y_1,$ $Y_2$ be two real-valued random variables. The excursion metric $E_m$ is given by
\begin{equation}
\label{Em:def:eq}
    E_m(Y_1,Y_2):=\int_\mathbb R \left(\mathbb{P}(Y_1>u)+\mathbb{P}(Y_2>u)
-2\mathbb{P}(Y_1>u,Y_2>u)\right) m(du).
\end{equation}
\end{definition}
In order to understand when functional $E_m$ is indeed a metric on the space $L^0(\Omega,{\cal F}, \mathbb{P})$ of  all random variables defined on this probability space, we present several equivalent forms of \eqref{Em:def:eq}.

In the sequel, we will need the distribution function of measure $m$ given by
 $$F_U(x):=\int_{-\infty}^xm(du), x\in\mathbb R,\quad  F_U(x-)=\lim_{y\to x-}F_U(y),$$
and the notation
$$\Delta_{Y_1,Y_2}(u):=\mathbb{P}(Y_1>u)+\mathbb{P}(Y_2>u)
-2\mathbb{P}(Y_1>u,Y_2>u).$$
It clearly holds  $E_m(Y_1,Y_2)=\int_\mathbb R \Delta_{Y_1,Y_2}(u) m(du).$
Denote by $Y_1\vee Y_2$ ($Y_1\wedge Y_2$) the maximum (minimum, resp.) of the random variables $Y_1$  and $Y_2$.
\begin{remark}
Let $F_1$ and $F_2$ be the distribution functions of $Y_1$ and $Y_2,$ respectively, and $C$ be the copula of $(Y_1,Y_2).$ Then  one writes
\begin{align}
\label{delta:eq0}\Delta_{Y_1,Y_2}(u)&=\mathbb{P}(Y_1\vee Y_2> u)-\mathbb{P}(Y_1\wedge Y_2> u)
=\mathbb{P}(Y_1\wedge Y_2\leq u)-\mathbb{P}(Y_1\vee Y_2\leq  u)   \\
&=\mathbb{P}(Y_1\leq u)+\mathbb{P}(Y_2\leq u)-2\mathbb{P}(Y_1\leq u,Y_2\leq u)  \nonumber \\
&=F_1(u)+F_2(u)-2\mathbb{P}(Y_1\vee Y_2\leq u)  =F_1(u)+F_2(u)-2 C(F_1(x),F_2(x)), \label{delta:eq2}
\end{align}
where the last relation follows from Sklar's theorem \cite{Sklar59}.
Moreover, it follows
\begin{align*}
    \Delta_{Y_1,Y_2}(u)&=\frac{1}{2}\mathbb{P}(Y_1>u)+\frac{1}{2}\mathbb{P}(Y_2>u)-\mathbb{P}(Y_1>u,Y_2>u)\\
    &+\frac{1}{2}\mathbb{P}(Y_1\leq u)+\frac{1}{2}\mathbb{P}(Y_2\leq u)-\mathbb{P}(Y_1\leq u,Y_2\leq u)\\
    &=1-\mathbb{P}((Y_1-u)(Y_2-u)\geq 0)=\mathbb{P}((Y_1-u)(Y_2-u)< 0)\in [0,1].
\end{align*}
Thus, it follows the relation
$0\leq E_m(Y_1,Y_2)\leq m(\mathbb R).$ \end{remark}

Without loss of generality, we may thus divide both sides of the last inequality by $ m(\mathbb R)$ and consider $m$ to be a probability measure which yields
$0\leq E_m(Y_1,Y_2)\leq 1.$
Let $U$ be a random variable with probability law $m$ which is independent of $Y_1$, $Y_2$. It can be interpreted as a random excursion level which we choose to build the metric $E_m$.
\begin{lemma}\label{lemmFormEm}
Let $m$ be a probability measure on $\mathbb R$ with c.d.f.  $F_U(x)=\int_{-\infty}^x m(dy),$ $x\in\mathbb R.$
Then
 \begin{align}
 \label{Em:eq1}
 E_m(Y_1,Y_2)&=\mathbf {E}|F_U(Y_2-)-F_U(Y_1-)|.
 \end{align}
\end{lemma}
\begin{proof}
 We have from relation \eqref{delta:eq0} that
\begin{align*}
    E_m(Y_1,Y_2)&=\int_{\mathbb R}\mathbf {E} \left(\mathbbm{1}\{u<Y_1\vee Y_2\}-\mathbbm{1}\{u<Y_1\wedge Y_2\}\right) m(du)\\
    &=\mathbf {E} \left(F_U(Y_1\vee Y_2-)-F_U(Y_1\wedge Y_2-) \right)=\mathbf {E}|F_U(Y_2-)-F_U(Y_1-)|.
 \end{align*}
\end{proof}
Equation \eqref{Em:eq1} can be interpreted as a probability that $U$ separates $Y_1$ and $Y_2$:
$$
E_m(Y_1,Y_2)=\mathbb{P}( Y_1\wedge Y_2 \le U<  Y_1\vee Y_2 ).
$$
Seen this way, $E_m$ coincides with the {\it separation (pseudo)metric} introduced by M. Taylor \cite{Taylor84}. The first part of the following corollary is also contained in \cite[Remark 1]{Taylor84}:

\begin{corollary} \label{corEm}
If $F_U$ is continuous then it holds $F_U(x-)=F_U(x),x\in\mathbb R,$ and
\begin{equation} E_m(Y_1,Y_2)=\mathbf {E}|F_U(Y_2)-F_U(Y_1)|. \label{eqE_m1}
 \end{equation}
If, in addition, $m$ is absolutely continuous with density $\psi$
then
\begin{equation*}
 E_m(Y_1,Y_2)=\mathbf {E}\left|\int_{Y_1}^{Y_2}\psi(u)du\right|=\frac{1}{2} \mathbf {E} \left[  |Y_1-Y_2|\int_{-1}^{1}\psi\left(s\frac{|Y_1-Y_2|}{2}+\frac{Y_1+Y_2}{2}\right)ds \right].
 \end{equation*}
\end{corollary}

The next theorem was also proven (under different assumptions) in \cite[Theorem 2]{Taylor84}:

\begin{theorem} \label{thm:metric}
Let $X_S$ be the space of random variables with support $S\subseteq\mathbb R.$ If $F_U$ is strictly increasing on $S,$ then $E_m$ is a metric on $X_S\times X_S.$
\end{theorem}
\begin{proof}
The symmetry of $E_m$ is trivial. The triangle inequality  follows for arbitrary $Y_1,Y_2,Y_3\in X_S$ from
\begin{align*}
    &E_m(Y_1,Y_2)= \mathbf {E}|F_U(Y_2-)-F_U(Y_1-)|\\
    &\leq \mathbf {E}|F_U(Y_2-)-F_U(Y_3-)|+\mathbf {E}|F_U(Y_3-)-F_U(Y_1-)|=E_m(Y_1,Y_3)+E_m(Y_3,Y_2).
\end{align*}
Let for some $Y_1,Y_2\in X_S:$ $\mathbf {E}|F_U(Y_2-)-F_U(Y_1-)|=0,$ then $F_U(Y_1-)=F_U(Y_2-)$ a.s. Since $F_U$ is strictly increasing on $S,$ $\mathbb{P}(Y_1=Y_2)=1$. Thus, $E_m$ is a metric.
\end{proof}
It can be easily shown  that $E_m$ given in \eqref{eqE_m1}  (with a specific choice of $U:(0,1) \to \mathbb R$ being a homeomorphism) coincides with the metric $d_{h,p}$ for $p=1$, $h=U$ from \cite{Taylor85} which metrizes the weak convergence in the space of distribution functions.

\section{Optimal choice of a weighting measure $m$} \label{sectOptChoice}

In this section, we assume that random variables $Y_1$ and $Y_2$ are absolutely continuously distributed with support $S\subseteq\mathbb R$, cumulative distribution functions (c.d.f.'s) $F_{1}$, $F_2$ and copula $C(\cdot,\cdot)$.
 Let $D$ be the space of all probability measures on $\mathbb R$.
\begin{theorem}
The maximum
 $$\max_{m\in D} E_m(Y_1,Y_2)$$
 is attained at  a measure $m$ with c.d.f. $F_U^*(x)=\mathbbm{1}\{u^*\leq x\},$ where $$u^*=\arg\max_{u\in\mathbb R}(F_1(u)+F_2(u)-2 C(F_1(u),F_2(u))).$$

If additionally $Y_1\stackrel{d}{=}Y_2$ then
\begin{equation}\label{eq:maxEm}
\max_{m\in D} E_m(Y_1,Y_2)=2\max_{x\in[0,1]}(x-C(x,x)),
\end{equation}
whereas $u^*=F_1^{-1}(x^*)$ and  $x^*=\arg\max_{x\in[0,1]} (x-C(x,x)).$
\end{theorem}
\begin{proof}
Recall from \eqref{delta:eq2} that
    $\Delta_{Y_1,Y_2}(u)=F_1(u)+F_2(u)-2 C(F_1(u),F_2(u))\in [0,1].$
Therefore, there exists, not necessarily unique,
$$u^*=\arg \max_{u\in \mathbb R} \left[F_1(u)+F_2(u)-2 C(F_1(u),F_2(u))\right].$$
Then
$$
E_m(Y_1,Y_2)=\int_\mathbb R \Delta_{Y_1,Y_2}(u)  \, m(du)\leq F_1(u^*)+F_2(u^*)-2C(F_1(u^*),F_2(u^*))=\int_\mathbb R \Delta_{Y_1,Y_2}(u) \delta_{u^*}(du).
$$
If $Y_1\stackrel{d}{=}Y_2$ then $F_1=F_2$ and hence
$$2 \max_{u\in \mathbb R} \left[F_1(u)-C(F_1(u),F_1(u))\right]= 2 \max_{x\in [0,1]} \left[x-C(x,x)\right],$$
since $F_1$ is non-decreasing.
\end{proof}

\begin{corollary} \label{corMaxValofE_m}
Let the random vector $(Y_1,Y_2)$ have a density function $p:\mathbb R^2\to\mathbb R_+$ with $Y_1\stackrel{d}{=}Y_2$.  Then $u^*$ satisfies the equation
$$\int_{u^*}^{+\infty} p(y,u^*)dy=\int_{-\infty}^{u^*} p(u^*,y)dy.$$
If  $p$ is additionally unimodal  and symmetric around its mode  $(\mu,\mu)$ then $u^*=\mu,$  $x^*=0.5,$
and $$\max_{m\in D} E_m(Y_1,Y_2)=1-2C(0.5,0.5).$$
\end{corollary}
\begin{proof} It holds
\begin{align*}
    &\max_{x\in[0,1]}(x-C(x,x))=\max_{u\in\mathbb R}(F(u)-C(F(u),F(u)))\\
    &=\max_{u\in \mathbb R}\left(\int_\mathbb R\int_{-\infty}^u p(y_1,y_2)dy_1dy_2-\int_{-\infty}^u\int_{-\infty}^u p(y_1,y_2)dy_1dy_2\right)\\
    &=\max_{u\in \mathbb R}\int_u^{+\infty}\int_{-\infty}^u p(y_1,y_2)dy_1dy_2.
\end{align*}
The maximum is reached on an extremal point $u$ such that
$$\frac{d}{du}\int_u^{+\infty}\int_{-\infty}^u p(y_1,y_2)dy_1dy_2=0,$$ or
$$\int_u^{+\infty} p(y,u)dy=\int_{-\infty}^{u} p(u,y)dy.$$

Now let $p$ be unimodal and symmetric around its mode $(\mu,\mu),$ then
\begin{align*}
    &\int_\mu^{+\infty} p(y,\mu)dy=\int_{-\infty}^\mu p(y,\mu)dy=\frac{1}{2} \int_\mathbb R p(y,\mu)dy,\\
    &\int_{-\infty}^\mu p(\mu,y)dy=\int_\mu^{+\infty} p(\mu,y)dy=\frac{1}{2} \int_\mathbb R p(\mu,y)dy=\frac{1}{2} \int_\mathbb R p(y,\mu)dy.
\end{align*}
Thus, the maximum is reached at $u^*=\mu.$
\end{proof}

\begin{remark}
Although the maximum of $E_m$ is reached on $m=\delta_{\{u^*\}},$
 $E_{\delta_{\{u^*\}}}$ is not a metric but just a pseudo metric.

\end{remark}

Which choice of  $m$ is preferable to keep $E_m$ a metric which is relatively easy to compute, infer and interpret?
 We know that $F_U$ should be strictly increasing on support $S.$  If we take $F_U=F_1$ then  it follows from the proof of Lemma \ref{lemmFormEm} that
\begin{align}
E_{F_1}(Y_1,Y_2) &=    \mathbf {E} F_1 (Y_1\vee Y_2)-\mathbf {E} F_1 (Y_1\wedge Y_2) \nonumber \\
    &=2\mathbf {E} F_1 (Y_1\vee Y_2)-\mathbf {E} F_1 (Y_1)-\mathbf {E} F_1 (Y_2)=2\mathbf {E} F_1 (Y_1\vee Y_2)-\mathbf {E} F_1 (Y_2)-\frac{1}{2}, \label{EF1:eq1}
\end{align}
since it holds $ F_1 (Y_1)\sim U(0,1)$ with expected value $1/2$. $E_{F_1}$ is a metric on the space of all random variables with absolutely continuous distributions on support $S$.

 If additionally $Y_1\stackrel{d}{=}Y_2,$ we get
$$E_{F_1}(Y_1,Y_2)=2\mathbf {E} F_1(Y_1\vee Y_2)-1=2\mathbf {E} (F_1(Y_1)\vee F_1(Y_2))-1$$
from equation \eqref{EF1:eq1} since $F_1$ is strictly increasing. It follows from \cite[p. 68]{DurSem16} that $$C(x,x)=\mathbb{P}( F_1(Y_1) \vee F_1(Y_2) \le x  ), \quad x\in[0,1].$$
Using relation  \eqref{delta:eq2} one writes after the substitution $x=F_1(u)$ that
 \begin{equation}\label{eq:E_mExp}
E_{F_1}(Y_1,Y_2)=2\int_{\mathbb R}\left[F_1(u)-C(F_1(u),F_1(u))\right]d F_1(u)=2\int_{0}^1\left[x-C(x,x)\right]dx=1-2 \int_0^1 C(x,x) \, dx.
\end{equation}
Since the term
$
2\int_{0}^1\left[x-C(x,x)\right]dx
$ is equal to the {\it Gini coefficient} of the {\it  Lorenz curve} $$\{  (x, C(x,x)), \; x\in[0,1]  \}$$ in case of a convex $ \{  C(x,x), \; x\in[0,1]  \}$, we come to the following definition:

\begin{definition}
Let $L_{F_1}$ be a space of random variables  with absolutely continuous c.d.f. $F_1$. The metric $G=E_{F_1}$ given by
\begin{equation*}\label{eq:G}
G(Y_1,Y_2)=1-2 \int_0^1 C(x,x) \, dx, \quad Y_1, Y_2\in L_{F_1},
\end{equation*}
where $C$ is the copula of  $(Y_1, Y_2)$, is called a Gini metric on $L_{F_1}$.
\end{definition}
By definition, the Gini metric is  distribution--free: it takes only the dependence structure between $Y_1$ and $Y_2$ into account, but not the marginal distribution of $Y_1,Y_2$.

\begin{remark}
In case $Y_1\stackrel{d}{=}Y_2$, the maximum value \eqref{eq:maxEm} equals the $L^\infty([0,1])$--distance between the diagonals of $C$ and of the upper Fr\'{e}chet--Hoeffding bound $M_2(x,y)=\min\{x,y\},$ $x,y\in[0,1]$, cf. e.g. \cite[Theorem 1.7.3]{DurSem16}.  Namely,
\begin{equation}\label{eq:DiagLinf}
\sup_{m\in D}  E_{m}(Y_1,Y_2)=2\| M_2(x,x) -C(x,x)    \|_{L^\infty([0,1])}.\end{equation}
Similarly, relation \eqref{eq:E_mExp} yields $G(Y_1,Y_2)=2\| M_2(x,x)-C(x,x)\|_{L^1([0,1])}$. In other words, the Gini metric $G$ measures the $L^1$--deviation of the diagonal of the copula $C$ of  $(Y_1, Y_2)$ to the diagonal of the comonotonicity copula $M_2$.
\end{remark}

\begin{lemma}\label{lemm:BoundsG}
It holds $0\le G(Y_1,Y_2) \le 1/2$,  $Y_1, Y_2\in L_{F_1}$. The upper bound is attained whenever $Y_2=f(Y_1)$ a.s. for some decreasing function $f$ such that
$F_1(x)=1-F_1(f^{-1}(x))$, $x\in \mathbb R$.
\end{lemma}
\begin{proof}
Using the Fr\'{e}chet--Hoeffding bounds \cite[Theorem 1.7.3]{DurSem16}
$$
W_2(x,y)\le C(x,y)\le M_2(x,y), \quad x,y\in [0,1]
$$
with $W_2(x,y)=\max\{ 0, x+y-1 \}$  being a copula of linearly dependent random variables, one can easily calculate
$$
1/4\le \int_0^1 C(x,x) \, dx \le 1/2.
$$
 Thus, relation  \eqref{eq:E_mExp}  yields the  bounds $0\le G(Y_1,Y_2) \le 1/2$, whereas the upper bound is attained by  \cite[Theorem 2.5.13 (d)]{DurSem16} whenever there exists a decreasing function $f$ such that $Y_2=f(Y_1)$ a.s. Since we assume $Y_1\stackrel{d}{=}Y_2$ here, it can be only the case if
 $$F_1(x)=\mathbb{P}(Y_1\le x)= \mathbb{P}(f(Y_2)\le x)=\mathbb{P}(Y_2\ge f^{-1}(x))=1- \mathbb{P}(Y_2< f^{-1}(x))=1-F_1(f^{-1}(x))$$
 for any real $x$. In the latter relation, we used the absolute continuity of the distribution of $Y_2$.
\end{proof}
\begin{example}
The upper bound $1/2$ is attained by Gini metric in Lemma \ref{lemm:BoundsG} if  $Y_1+Y_2=\mu$ a.s. for some $\mu\in\mathbb R$  and  the distribution of $Y_1$ is symmetric about $\mu$:
$F_1(x)=1-F(\mu-x)$, $x\in \mathbb R$. To see this, just take $f(x)=\mu-x$ in  Lemma \ref{lemm:BoundsG}.
\end{example}
\begin{example}\label{ex:GiniIndependent}
The Gini distance between stochastically independent  random variables $Y_1, Y_2\in L_{F_1}$  is equal to $1/3$, since in this case $C(x,y)=xy$, $x,y\in[0,1],$ and thus
$G(Y_1,Y_2) = 1-2\int_0^1 x^2 \,dx=  1/3$.
\end{example}

To summarize, the choice $m=\mathbb{P}_{Y_1}$ for an absolutely continuous law $\mathbb{P}_{Y_1}$ seems  natural to make $E_m$ a metric. Gini metric $G$ is easy to calculate and distribution--free (depending only on the copula $C$ of $(Y_1,Y_2)$, cf. relation \eqref{eq:E_mExp})  when $\mathbb{P}_{Y_1}=\mathbb{P}_{Y_2}$. Informally speaking, the excursion levels are chosen here according to the same law as  $\mathbb{P}_{Y_1}=\mathbb{P}_{Y_2}$ which makes the corresponding excursion sets non--empty (with positive probability) and representative for $Y_1, Y_2$.

\section{Prediction of random variables}\label{sectPredict}

In this section, we discuss the prediction of a value of random variable $X$ with continuous distribution function $F_X$ based on the set $\mathcal{X}_n:=(X_1,\ldots,X_n)$ of realizations of $X$  via the excursion metric metric $E_{F_X}$. Namely, we propose a predictor $\widehat{X}_\lambda:=g(\lambda,mathcal{X}_n),$ where $\lambda=(\lambda_1,\ldots, \lambda_d) \in \Lambda\subset\mathbb R^d$ is deterministic  and $g:\mathbb R^n\times \Lambda\to \mathbb R$, $d,n\in\mathbb N$, is a continuous measurable function such that the excursion metric is minimal:

\begin{equation}\label{eq:E_PredictFormal}
E_{F_X}(X, \widehat{X}_\lambda) \to \min_{\lambda\in \Lambda}.
\end{equation}

Here, the set of admissible parameters $\Lambda$ as well as the analytic form of $g(\lambda,\mathcal{X}_n)$ depend on  the law $\mathbb{P}_X$. For instance, the choice $g(\lambda,\mathcal{X}_n)=\sum_{j=1}^n \lambda_j X_j$ makes sense for infinitely divisible laws of $X$, whereas  $g(\lambda,\mathcal{X}_n)=\max_{j=1,\ldots,n} \lambda_j X_j$ might be a better choice for max--stable $X$. In both cases, we assume $d=n$. The set $\Lambda$ may incorporate additional constraints onto $\mathbb{P}_{\widehat X}$, for instance, $\widehat{X}_\lambda \stackrel{d}{=} X$.
Since $g$ and $F_X$ are continuous, the constraint $\widehat{X}_\lambda\stackrel{d}{=}X$ is equivalent to $F_X(\widehat{X}_\lambda)\stackrel{d}{=}F_X(X)\stackrel{d}{=}U,$ where $U\sim U(0,1).$ Under additional assumptions onto the joint probability law of  $(X, \mathcal{X}_n)$ and onto $g$, the set  $\Lambda_g:=\{\lambda\in \mathbb R^d :F_X(g(\lambda,\mathcal{X}_n))\stackrel{d}{=}U\}$ is a manifold in $\mathbb R^d$. Unfortunately, the analytic form of $\Lambda_g$ can be found only in specific cases when the pre-knowledge of the distribution of $(X,\mathcal{X}_n)$ is available, such as in the Gaussian or $\alpha$--stable case, cf. \cite{Dasetal22}.

Should our prediction be law-preserving (i.e.,  $\widehat{X}_\lambda \stackrel{d}{=} X$), it holds  $E_{F_X}=G$, and   the optimization problem
$G(X, \widehat{X}_\lambda) \to \min_{\lambda\in \Lambda}$
with $\Lambda=\Lambda_g$ rewrites using \eqref{eq:E_mExp} as
\begin{equation}\label{eq:E_PredictFormal_Copula}
\int_0^1 C _{X, \widehat{X}_\lambda}(x,x) \,dx  \to \max_{\lambda\in \Lambda_g},
\end{equation}
where $C _{X, \widehat{X}_\lambda}$ is the copula of $(X, \widehat{X}_\lambda)$. In order to avoid a tricky statistic assessment of copulas, we use however the following  forms of prediction which are motivated by  \eqref{EF1:eq1}   and  require only expectations to be inferred into:

\begin{definition}
The excursion predictor $\widehat{X}_\lambda$ is given by  $\widehat{X}_\lambda=g(\hat{\lambda},\mathcal{X}_n)$,  where
\begin{equation}
    \label{unconst:eq1}
    \hat{\lambda}:=\arg \min_{\lambda\in \Lambda}\left[2\mathbf {E} F_X (X\vee \widehat{X}_\lambda)-\mathbf {E} F_X (\widehat{X}_\lambda)\right]
\end{equation}
in general case, or
 \begin{equation}
    \label{const:eq2}
    \hat{\lambda}:=\arg \min_{\lambda\in \Lambda_g}\left\{\mathbf {E} F_X (X\vee \widehat{X}_\lambda)\right\}
\end{equation}
in case of law-preserving prediction $\widehat{X}_\lambda\stackrel{d}{=}X$.
\end{definition}

Let us consider the law-preserving case in more detail. If the analytic form of $ \Lambda$ is given explicitly but $ \Lambda_g$ is hardly available, we modify the minimization functional in \eqref{const:eq2} by adding a term which penalizes a  difference between the law of $Y_1=F_X(\widehat{X}_\lambda)$ and $Y_2\sim U(0,1)$:

\begin{equation}
    \label{const:eq3}
    \hat{\lambda}:=\arg \min_{\lambda\in \Lambda}\left\{\mathbf {E} F_X (X\vee \widehat{X}_\lambda)+\gamma \rho^2(F_{Y_1},F_{Y_2})\right\},
\end{equation}
where $\gamma>0$ is a penalty weight and $\rho$ is an arbitrary (but handy) metric  on the space of continuous distribution functions of random variables.
 For simplicity reasons, we use the {\it 2-Wasserstein distance }
    $$\rho(F_1,F_2)=\left(\int_0^1 [F^{-1}_1(x)-F^{-1}_2(x) ]^2dx\right)^{1/2}$$
   between two c.d.f.'s $F_1$ and $F_2$ with quantile functions  $F^{-1}_1$ and  $F^{-1}_2$, respectively.

    In the case $Y_2\sim U(0,1)$ we have  $F_{Y_2}^{-1}(x)=x$,  $x\in[0,1]$. Hence, the squared 2-Wasserstein distance equals
    \begin{equation}
    \label{eq:Wasserstn}
    \rho^2(F_{Y_1},F_{Y_2})=\int_0^1 x^2 dx +\int_0^1 y^2dF_{Y_1}(y)-\int_{0}^1 y d F_{Y_1}^2(y)=\frac{1}{3}+\mathbf {E} Y_1^2-\mathbf {E} [Y_1\vee Y],
    \end{equation}
     where $Y$ is an independent copy of $Y_1.$ The latter relation holds since $\mathbb{P}(Y_1\vee Y\le y)=  F_{Y_1}^2(y)$, $y\in \mathbb R$.
Due to $Y_1=F_X(\widehat{X}_\lambda)\in [0,1]$ a.s., it holds  $\mathbf {E} [Y_1\vee Y]\ge \mathbf {E} Y_1^2$. Thus, the minimization problem \eqref{const:eq3}  rewrites in an equivalent form:
\begin{equation}
    \label{eq:FinalPreidct}
    \hat{\lambda}:=\arg \min_{\lambda\in \Lambda}\left\{\mathbf {E} F_X (X\vee \widehat{X}_\lambda)+\gamma \left[  \mathbf {E} F_X^2(\widehat{X}_\lambda)- \mathbf {E} [F_X(\widehat{X}_\lambda)\vee Y]   \right]       \right\},
\end{equation}
where $Y$ is an independent copy of $F_X(\widehat{X}_\lambda)$.
    Compared with formulation \eqref{const:eq2}, the new prediction method   \eqref{eq:FinalPreidct} does not require an explicit knowledge of $\Lambda_g$, but it realizes the constraint
    $\widehat{X}_\lambda\stackrel{d}{=}X$ only in approximation form: $\rho(F_X, F_{\widehat{X}_\lambda})\le \varepsilon$ for some small $\varepsilon>0$.

    Sometimes it is advantageous to use the integration by parts in  \eqref{eq:Wasserstn} and write
   \begin{equation}
    \label{eq:Wasser1}
    \rho^2(F_{Y_1},F_{Y_2})=\frac{1}{3}+ \int_0^1 F_{Y_1}(y) \left[ F_{Y_1}(y)-2y\right]\, dy
  \end{equation}
  which allows for an equivalent reformulation
   \begin{equation}
   \label{eq:FinalPreidct1}
    \hat{\lambda}:=\arg \min_{\lambda\in \Lambda}\left\{\mathbf {E} F_X (X\vee \widehat{X}_\lambda)+\gamma  \int_0^1 F_{Y_1}(y) \left[ 2y-F_{Y_1}(y)\right]\, dy     \right\}
\end{equation}
of the problem     \eqref{eq:FinalPreidct}.

To summarize, prediction approach \eqref{unconst:eq1} will be used for unconstrained prediction of a random variable $X$ with an absolutely continuous c.d.f. $F_X$ based on its realizations $\mathcal{X}_n=(X_1,\ldots,X_n)$. For the law--preserving prediction, approaches  \eqref{const:eq2}  or \eqref{eq:FinalPreidct} will be used depending on whether the restrained parameter set $\Lambda_g$ is given explicitly or not.

\section{Existence of a solution}\label{sect:Exist}

A solution to the above optimization problems exists on compact parametric sets due to the continuity of the corresponding target functionals:

\begin{theorem}\label{thm:ExistSol}
Let the joint distribution of the random vector $(X,\mathcal{X}_n)$ be absolutely continuous with respect to the Lebesgue measure on $\mathbb R^{n+1}$. Introduce the following assumptions:
\begin{enumerate}
\item[\rm (i)] $\Lambda$ (or $\Lambda_g$, respectively) is a compact in $\mathbb R^n$.
\item[\rm (ii)] The copula diagonal $C _{X, \widehat{X}_\lambda}(x,x)$   of $(X, \widehat{X}_\lambda)$ is continuous on $\Lambda$ (or $\Lambda_g$, respectively)  uniformly w.r.t. $x\in[0,1]$.
\item[\rm (iii)] For each $\lambda\in\Lambda$, $\widehat{X}_\lambda$ has an absolutely continuous distribution with density   $p_{\widehat{X}_\lambda}$   such that the map $p_{\widehat{X}_\lambda}:\Lambda\to L^1(\mathbb R)$ is continuous  on $\Lambda$ w.r.t. the $L^1$--norm.
\end{enumerate}
If the conditions {\rm (i)-(ii)} hold then there exists a solution to the problem \eqref{const:eq2}. If the conditions {\rm (i)-(iii)} hold then there exists a solution to the problems \eqref{unconst:eq1} and \eqref{eq:FinalPreidct1}.
\end{theorem}
\begin{proof}
To show the existence of a solution, it is sufficient to assume (i) and show that the target functional $\Phi(\lambda)$ to be maximized or minimized is continuous on $\Lambda_g$ or $\Lambda$.

In case of the problem \eqref{const:eq2}, we have
$$
\Phi(\lambda)=\int_0^1 C _{X, \widehat{X}_\lambda}(x,x) \,dx, \quad \lambda\in\Lambda_g
$$
with regard to \eqref{eq:E_PredictFormal_Copula}. Condition (ii) ensures the continuity of $\Phi$ on $\Lambda_g$ in view of the corresponding theorem for the continuity of integrals with parameters.

For the problem \eqref{unconst:eq1}, we have
$$
\Phi(\lambda)=2-2\int_0^1 C _{X, \widehat{X}_\lambda}(x,x) \,dx  -\mathbf {E} F_X (\widehat{X}_\lambda), \quad \lambda\in\Lambda
$$
together with
$$
\mathbf {E} F_X (\widehat{X}_\lambda)=\int_0^1 \mathbb{P}( F_X (\widehat{X}_\lambda)>y) dy=1- \int_0^1 F_{\widehat{X}_\lambda} ( F_X^{-1}(y)) dy,
$$
where $ F_{\widehat{X}_\lambda}$ is the c.d.f. of the predictor $\widehat{X}_\lambda$.
The latter relation holds since the c.d.f. $F_X$ is strictly increasing on  $S=\mbox{supp} (X)$. Due to the absolute continuity of the distribution of
$\widehat{X}_\lambda$, the function $F_{\widehat{X}_\lambda} ( F_X^{-1}(y))$ is continuous on $[0,1]$ for each $\lambda\in\Lambda$.
Moreover, for any sequence $\{ \lambda_n\} \subset \Lambda$ with $\lambda_n\to\lambda_0\in\Lambda$ as $n\to\infty$  we have
$$
\sup_{x\in S} \left| F_{\widehat{X}_{\lambda_n}}(x)-  F_{\widehat{X}_{\lambda_0}}(x) \right|=\sup_{x\in \mathbb R} \left|  \int_{-\infty}^x \left[ p_{\widehat{X}_{\lambda_n}}(y)-  p_{\widehat{X}_{\lambda_0}}(y)   \right] dy \right|\le \int_\mathbb R  \left| p_{\widehat{X}_{\lambda_n}}(y)-  p_{\widehat{X}_{\lambda_0}}(y) \right| dy\to 0
$$
as $n\to \infty$ by assumption (iii) which means the continuity of $ F_{\widehat{X}_\lambda} ( F_X^{-1}(y))$ on $ \Lambda$ uniformly w.r.t. $y\in[0,1]$.
The application of the theorem on the continuity of integrals with parameters finishes the proof.

In the problem \eqref{eq:FinalPreidct1},
the target functional rewrites
$$
\Phi(\lambda)=2-2\int_0^1 C _{X, \widehat{X}_\lambda}(x,x) \,dx  +\gamma  \int_0^1 F_{Y_1}(y) \left[ 2y-F_{Y_1}(y)\right]\, dy    , \quad \lambda\in\Lambda.
$$
Similarly to the previous case, it is not difficult to show that the integrand  $ F_{Y_1}(y) \left[ 2y-F_{Y_1}(y)\right]$  is  continuous on $\Lambda$ uniformly w.r.t. $y\in[0,1]$ provided that condition (iii) holds true.
\end{proof}

\begin{remark}
Condition (iii) of Theorem \ref{thm:ExistSol} means that for any sequence $\{ \lambda_n\} \subset \Lambda$ with $\lambda_n\to\lambda_0\in\Lambda$ as $n\to\infty$
$$d_{TV}(\widehat{X}_{\lambda_n},\widehat{X}_{\lambda_0})=\frac{1}{2}  \int_\mathbb R  \left| p_{\widehat{X}_{\lambda_n}}(y)-  p_{\widehat{X}_{\lambda_0}}(y) \right| dy
=\frac{1}{2}   \left\|  p_{\widehat{X}_{\lambda_n}}-  p_{\widehat{X}_{\lambda_0}} \right\|_1\to 0, \quad n\to\infty,
$$
where   $d_{TV}$ is the total variation distance and $\|  \cdot \|_1$ is the norm in $L^1(\mathbb R)$. It implies that  $\widehat{X}_{\lambda_n} \to \widehat{X}_{\lambda_0}$ in total variation as $n\to\infty$.
\end{remark}
Let us give some examples of the compacts $\Lambda_g\subset \mathbb R^n$. In what follows, the random vector $(X,\mathcal{X}_n)$ will have a joint $\alpha$--stable distribution for some
 $\alpha\in(0,2]$, hence it is natural to consider the linear predictor $\widehat{X}_\lambda=\sum_{j=1}^n \lambda_j X_j$.
\begin{example}\label{ex:Lambda}

\begin{enumerate}
\item If $(X,\mathcal{X}_n)$ is a Gaussian random vector with marginal distribution $N(\mu, \sigma^2)$ and $\Sigma$ is the covariance matrix of   $\mathcal{X}_n$
then the manifold $\Lambda_g$ is an ellipsoid of dimension $n-1$ given by
$$
\Lambda_g=\left\{  \lambda=(\lambda_1,\ldots, \lambda_n)\in \mathbb R^n:  \lambda^\top \Sigma \lambda=\sigma^2, \quad \sum_{j=1}^n \lambda_j=1 \right\}.
$$
\item If $(X,\mathcal{X}_n)$ is a subgaussian random vector with  stability index $\alpha\in(0,2)$  and i.i.d. standard Gaussian components then it follows from \cite[p. 80-81]{SamTaq94} that
 $\Lambda_g$ is a unit sphere $S^{n-1}$  in $\mathbb R^n$ given by
$$
\Lambda_g=\left\{  \lambda=(\lambda_1,\ldots, \lambda_n)\in \mathbb R^n:   \sum_{j=1}^n \lambda_j^2=1 \right\}.
$$
\item If $(X,\mathcal{X}_n)$ is a symmetric $\alpha$--stable random vector with  stability index $\alpha\in(0,2)$, scale parameter $\sigma=1$ of the marginal distributions   and spectral measure $\Gamma$ of   $\mathcal{X}_n$ then it follows from \cite[p. 73]{SamTaq94} that
$$
\Lambda_g=\left\{  \lambda=(\lambda_1,\ldots, \lambda_n)\in \mathbb R^n:   \int_{S^{n-1}} |\langle  \lambda, s\rangle|^\alpha \, \Gamma(ds)=1 \right\}.
$$
The dominated convergence theorem helps showing that $\Lambda_g$ is a  closed set.
For $\alpha\in[1,2)$, rewrite the constraint in $\Lambda_g$ as
$$
| \lambda | =  h_K^{-1} (u), \quad u=\lambda/|\lambda|\in S^{n-1}  ,
$$
where  $|\cdot|$ is the Euclidean norm in $\mathbb R^n$ and
$$
h_K (u)= \left( \int_{S^{n-1}} |\langle  \lambda, s\rangle|^\alpha \, \Gamma(ds)\right)^{1/\alpha}
$$
is the support function of a convex set $K= \Gamma^{1/\alpha} (S^{n-1}) \mathbf {E} _\alpha [-\eta, \eta]  $ named $L_\alpha$-{\it zonoid}.
Here $\eta$ is a random vector on $S^{n-1}$ distributed according to $\Gamma (\cdot)/ \Gamma (S^{n-1})$ and $\mathbf {E}_\alpha  [-\eta, \eta]$ is the Firey $\alpha$--expectation of the random segment $ [-\eta, \eta]$, cf. e.g. \cite{Molch09}.
If $K$ is full-dimensional  (which is e.g. the case if $\Gamma$ has a density w.r.t. to the surface area measure on $S^{n-1}$ which is bounded away from zero everywhere on $S^{n-1}$) we have $\inf_{u\in S^{n-1}} h_K(u)>0$ and thus $\Lambda_g$ is bounded, hence a compact.
\end{enumerate}
\end{example}
\begin{example}
Show that  conditions (i)-(iii) of Theorem  \ref{thm:ExistSol}  are satisfied if $(X,\mathcal{X}_n)$ is a Gaussian random vector with marginal distribution $N(0,1)$. For any $\lambda\in \Lambda_g$, it holds $\widehat{X}_\lambda\sim N(0,1)$.  Condition (i) was shown in Example \ref{ex:Lambda}. Let us check condition (ii).
In view of \cite{Dasetal22}, the copula diagonal writes
$$
C _{X, \widehat{X}_\lambda}(x,x)=x^2+ \frac{1}{2\pi} \int_{0}^{\sin^{-1}\left(\rho_\lambda \right)} \exp\left(- ( \varphi^{-1}(x))^{2}\frac{1-\sin\left(\theta\right)}{\cos^{2}\left(\theta\right)}\right)d\theta,
$$
where $  \varphi^{-1}(x)$ is the quantile function of $N(0,1)$ and  $\rho_{\lambda} = \mathrm{Corr}\left(X,\widehat{X}_\lambda \right)=\sum_{j=1}^n \lambda_j \mathrm{Cov}\left(X,{X}_j\right)$.
Since the exponential function under the integral is nonnegative and bounded from above by one, we get
$$
\left|  C _{X, \widehat{X}_{\lambda_1}}(x,x)-C _{X, \widehat{X}_{\lambda_2}}(x,x)  \right|\le   \frac{1}{2\pi} \left|   \sin^{-1}\left(\rho_{\lambda_1}\right) - \sin^{-1}\left(\rho_{\lambda_2}\right)   \right|, \quad \lambda_!, \lambda_2 \in \Lambda_g
$$
uniformly on $x\in [0,1]$ which shows the uniform continuity of $C _{X, \widehat{X}_\lambda}(x,x)$ on $\Lambda_g$.
To show (iii), let the covariance matrix $\Sigma$ of  $\mathcal{X}_n$ be positive definite. Then $\widehat{X}_{\lambda}\sim N(0, \lambda^\top \Sigma \lambda)$ where $\lambda^\top \Sigma \lambda>0$ for all $\lambda\neq 0$, hence it has a Gaussian density which is continuous on $\mathbb R^n\setminus\{0\}$ in the $L^1$--norm. To see this, just use the multivariate mean value theorem for this density with respect to $\lambda$.

\end{example}

The next result shows that it is sufficient to consider bounded spaces $\Lambda$ or $\Lambda_g$ in minimization problems \eqref{unconst:eq1}, \eqref{const:eq2}, \eqref{eq:FinalPreidct}, and \eqref{eq:FinalPreidct1}. If these spaces are additionally closed, the existence of a solution is guaranteed by Theorem  \ref{thm:ExistSol}. For instance, so is  often the choice
$$
\Lambda=\{  \lambda\in \mathbb R^n: |\lambda|\le M \} \mbox{ or }  \Lambda=\{  \lambda\in \mathbb R_+^n: |\lambda|\le M \}
$$
for a suitable $M>0$.
Introduce the notation
$$
\Phi_1(\lambda):=\mathbf {E} F_X (X\vee \widehat{X}_\lambda)  ,
$$
$$
\Phi_2(\lambda):=2\mathbf {E} F_X (X\vee \widehat{X}_\lambda)-\mathbf {E} F_X (\widehat{X}_\lambda)   ,
$$

$$
\Phi_3(\lambda):=\mathbf {E} F_X (X\vee \widehat{X}_\lambda) +\gamma \left[ \mathbf {E} F_X^2(\widehat{X}_\lambda)- \mathbf {E} [F_X(\widehat{X}_\lambda)\vee Y] \right]
$$
for the target functionals in minimization problems \eqref{const:eq2}, \eqref{unconst:eq1}, \eqref{eq:FinalPreidct} or \eqref{eq:FinalPreidct1}, respectively.

\begin{proposition} \label{thm:Bounded}
Assume that there exists $\lambda _0 \in \Lambda_g$ or $\Lambda$ such that $ \Phi_j(\lambda_0)<1$, $j=1,2,3$.
Let $\widehat{X}_{\lambda_k} \stackrel{P}{\longrightarrow} + \infty$ as $k\to\infty$ for any sequence $\{\lambda_k\}\subset \Lambda$  such that $|\lambda_k| \to + \infty$. Then there exists $M>0$ such that
 $$
  \min_{\lambda\in\Lambda_g  }\Phi_1(\lambda)= \min_{\lambda\in\Lambda_g: |\lambda|\le M }\Phi_1(\lambda), \quad
 \min_{\lambda\in\Lambda  }\Phi_j(\lambda)= \min_{\lambda\in\Lambda: |\lambda|\le M }\Phi_j(\lambda), \quad j=2,3.
 $$
\end{proposition}
\begin{proof}
Sequences of random variables $\{  F_X (\widehat{X}_{\lambda_k})  \}$,  $\{  F_X^2 (\widehat{X}_{\lambda_k})  \}$, $\{  F_X (X\vee \widehat{X}_{\lambda_k})  \}$,  $\{  F_X(\widehat{X}_{\lambda_k})\vee Y  \}$
are uniformly integrable since they are a.s.  bounded by zero and one. Hence, their expectations tend to one as  $|\lambda_k| \to + \infty$ by properties of a c.d.f. Then it holds
$$
\Phi_j(\lambda_k) \to 1, \quad   |\lambda_k| \to + \infty, \quad j=1,2,3.
$$
Take $M>0$ such that $\Phi_j(\lambda_k) >  \Phi_j(\lambda_0)$ for all $k$ such that $|\lambda_k|>M$,   $j=1,2,3$.
The assertion is proven.
\end{proof}

\begin{example}  Assume that there exists a $\lambda_0 \in\Lambda_g$ or $\Lambda$ such that
\begin{enumerate}
\item  $ \widehat{X}_{\lambda_0}=X$ a.s. This is the case for some  prediction functions $g$ if $X=X_{j_0}$ a.s., $j_0\in\{ 1,\ldots,n\}$. Then it can be easily shown that
$ \Phi_1(\lambda_0)= \Phi_2(\lambda_0)=1/2$, $ \Phi_3(\lambda_0)=  1/2+\gamma/3<1$ for all $\gamma\in (0,3/2)$.
\item   $ \widehat{X}_{\lambda_0}$ and $X$ are stochastically independent.  This can be the case if $X,\mathcal{X}_n$ form an $m$--dependent sequence with $m<n$. Then it can be easily shown that
$ \Phi_1(\lambda_0)= 2/3$ (cf. Example \ref{ex:GiniIndependent}),
$$\Phi_2(\lambda_0)=1+\int_0^1 F_{Y_1}(x)(1-2x)dx <1$$ with $Y_1=F_X(\widehat{X}_{\lambda_0})$ if
$\int_0^1 F_{Y_1}(x)(1-2x)dx <0$,
$$ \Phi_3(\lambda_0)=  1+\int_0^1 F_{Y_1}(x)\left( (2\gamma-1) x - \gamma F_{Y_1}(x) \right) dx <1$$ for $\gamma \in (0, \gamma_1)$ where
 $\gamma_1= \frac{    \int_0^1 x F_{Y_1}(x) \, dx    }{  \int_0^1 F_{Y_1}(x)\left( 2 x - F_{Y_1}(x) \right) dx }$.
\end{enumerate}
Notice that in case of the minimization problem \eqref{eq:FinalPreidct} or \eqref{eq:FinalPreidct1} the choice of constant $\gamma>0$ matters. Only relatively small values of $\gamma$
are allowed in order to guarantee the existence of a solution.
\end{example}
\begin{example}
Condition $\widehat{X}_{\lambda_k} \stackrel{P}{\longrightarrow} + \infty$ as $k\to\infty$ for any sequence $\{\lambda_k\}\subset \Lambda$  such that $\lambda_k=( \lambda_k(1),\ldots,\lambda_k(n))$,  $|\lambda_k| \to + \infty$ is satisfied for
$\Lambda=\mathbb R^n_+$, $\widehat{X}_{\lambda_k} =\sum_{j=1}^n \lambda_k(j)X_j$ or $\widehat{X}_{\lambda_k}=\max_{j=1,\ldots, n} \lambda_k(j)X_j$ and a.s. nonnegative random variables  $X_j$, $j=1,\ldots,n$.

\end{example}

The question of  uniqueness of a solution $\lambda$ to problems  \eqref{unconst:eq1},  \eqref{const:eq2}  and \eqref{eq:FinalPreidct} cannot be resolved in such generality. As illustrated  in the Gaussian case \cite{Dasetal22}, it will require further specification of the dependence structure of observations $X_j$ within the set $\mathcal{X}_n$, of the statistic $g$ and parameter set $\Lambda$. This will be done for some specific classes of random variables in forthcoming research.

\section{Excursion-based extrapolation of stationary heavy--tailed random fields}\label{sectExtrapol}

In this section, we will use the prediction approach  \eqref{eq:FinalPreidct}  to extrapolate a real--valued strictly stationary  random field  $X=\{X(t),t\in \mathbb R^d\}$ with absolutely continuous (but  possibly heavy--tailed) marginal distribution $F_{\theta_0}.$ We assume that $F_{\theta_0}$ belongs to an appropriate parametric family of possible marginal distributions $\{ F_\theta, \theta\in\Theta\}$, $\Theta\subseteq \mathbb R^k$. This approach can be used, in particular, for heavy--tailed time series forecasting in insurance/finance ($d=1$, compare Section \ref{sect:Num})   or in image analysis ($d=2,3$) for upscaling of low-resolution 2D and 3D gray scale images (cf.  the so-called {\it super-resolution problem} \cite{Chang04,KimKwon10,Freeman11c.:markov,Dong16}).

Denote by $\mathbb Z_h=(h_1 \mathbb Z)\times \cdots \times (h_d \mathbb Z) $ the $d-$dimensional grid with mesh sizes $h=(h_1,\ldots,h_d)\in (0,+\infty)^d.$
Let $X$ be potentially observed at points $\mathbb{T}_0:=W_o\cap \mathbb Z_h,$ where $W_o\subset \mathbb R^d$ is a compact.  The observed values form a sample
 $\mathcal{X}_T:=\{X(t_j),t_j\in \mathbb{T}_0\}.$

  Let us predict the value $X(t)$ at a location $t \in \mathbb Z_h$, $t \not \in W_o$ from the knowledge of the so--called {\it forecast sample} $T_t:= \{t_1,\ldots,t_n\} \neq \emptyset,$ $T_f\subset \mathbb Z_h.$ The predictor $\widehat{X}_\lambda=g(\lambda,\mathbf{X}(T_t))$ with $\lambda\in \Lambda\subseteq\mathbb R^n$ and $\mathbf{X}(T_t):=(X(t_1),\ldots,X(t_n))^{\top}$ requires weights $\lambda$  to be a solution of minimization problems  \eqref{unconst:eq1},
 \eqref{eq:FinalPreidct} or \eqref{eq:FinalPreidct1}, i.e.,
 \begin{equation}\label{eq:minFctl_noConstraint}
\hat{\lambda}=\arg \min_{\lambda\in \Lambda}\left[2\mathbf {E}  \left[ F_{\theta_0} (X(t))\vee F_{\theta_0} ( \widehat{X}_\lambda) \right] - \mathbf {E} F_{\theta_0}(\widehat{X}_\lambda)  \right] ,
\end{equation}
 \begin{equation}\label{eq:minFctl}
\hat{\lambda}=\arg \min_{\lambda\in\Lambda}\left\{\mathbf {E}  \left[ F_{\theta_0} (X(t))\vee F_{\theta_0} ( \widehat{X}_\lambda) \right]+\gamma
\left[\mathbf {E} F_{\theta_0}^2(\widehat{X}_\lambda) -  \mathbf {E} [F_{\theta_0}(\widehat{X}_\lambda)\vee Y]  \right]   \right\} ,   \mbox{ or}
\end{equation}
 \begin{equation}\label{eq:minFctl1}
\hat{\lambda}=\arg \min_{\lambda\in\Lambda}\left\{\mathbf {E}  \left[ F_{\theta_0} (X(t))\vee F_{\theta_0} ( \widehat{X}_\lambda) \right]+\gamma
\left[  \int_0^1 F_{F_{\theta_0} ( \widehat{X}_\lambda)}(y) \left[ 2y-F_{F_{\theta_0} ( \widehat{X}_\lambda)}(y)\right]\, dy  \right]   \right\} ,
\end{equation}
respectively, due to strict monotonicity of  $F_{\theta_0}$, where $Y$ is an independent copy of $F_{\theta_0}(\widehat{X}_\lambda)$ and $F_{F_{\theta_0} ( \widehat{X}_\lambda)}$ is the c.d.f. of the random variable $F_{\theta_0} ( \widehat{X}_\lambda)$.
As already mentioned in Section \ref{sectPredict}, we use  $$\widehat{X}_\lambda=\lambda^{\top}\mathbf{X}(T_t)$$ for infinitely divisible $X$ and
  $$\widehat{X}_\lambda=\max_{j=1,\ldots, n}\lambda(j) X(t_j)$$ with $\lambda=(\lambda(1),\ldots,\lambda(n))^\top$ for max--stable $X$.

If the parameter $\theta_0$ is unknown, we assess it by a statistic $\widehat \theta$  in order to find $F_{\widehat \theta},$ which is a plug-in estimator of $F_{\theta_0}.$
Then, we substitute expectations in \eqref{eq:minFctl_noConstraint}-\eqref{eq:minFctl1} by the corresponding empirical moments.
The  prediction problems above get the form
\begin{equation}\label{eq:EmpMoment}
     \Phi(\lambda):=\sum_{j=1}^N   Q_j(\lambda)  \to \min_{\lambda\in\Lambda},
\end{equation}
where
\begin{equation}\label{eq:Q_unconstraint}
Q_j(\lambda):=2F_{\widehat \theta} (X(t+h_j))\vee   F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))- F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))
\end{equation}
for unconstrained prediction \eqref{eq:minFctl_noConstraint},
\begin{equation}\label{eq:Q}
Q_j(\lambda):=F_{\widehat \theta} (X(t+h_j))\vee   F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))+\gamma\left[ F^2_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))-F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j))) \vee Y_{j}\right]
\end{equation}
for the  (approximatively) law-preserving prediction \eqref{eq:minFctl}, and
\begin{align}
Q_j(\lambda):&=F_{\widehat \theta} (X(t+h_j))\vee   F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))  - \gamma F^2_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))  \nonumber  \\
&+\frac{\gamma}{N} \left[  F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))  +2 \sum_{i=1}^{j-1}
F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_i))) \vee   F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j))) \label{eq:Q1}
 \right]
\end{align}
for the law-preserving prediction variant \eqref{eq:minFctl1}, where the convention $\sum_{i=1}^0=0$ is used. Here
$T_t+h_j$, $j=1,\ldots, N$ with  $\{h_1,\ldots,h_N\}:=\{s\in \mathbb Z_h: s+T_t\cup \{t\}\subset  \mathbb{T}_0   \}$ are the so--called {\it learning samples}, and
 $Y_j$ are independent copies of $F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))$. In practice, the sample $\{Y_1,\ldots,Y_N\}$ can be obtained for each $\lambda$ by bootstrap, i.e. resampling of
\begin{equation}\label{eq:Bootstr}\{ F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_1))),\ldots, F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_N)))\}.\end{equation}

\begin{remark}
The extrapolation methods \eqref{eq:minFctl_noConstraint}-\eqref{eq:minFctl1}  with $\theta_0$ replaced by $\theta_t$ can be also used  for the extrapolation of non--stationary random fields $X$ if their marginal distributions  $F_{\theta_t}(x)=\mathbb{P}(X(t)\le x)$ are known in advance. In this case,  the empirical moments  \eqref{eq:Q_unconstraint}-\eqref{eq:Bootstr} in the problem \eqref{eq:EmpMoment} have to be rewritten with
$F_{\theta_t}$ in lieu of $F_{\widehat \theta}$.
\end{remark}

In order to find the minimum of $\Phi(\lambda)$, we use a subgradient descent. Assume that $F_\theta$ has a density $p_\theta.$
Since the marginal distribution of $X$ is absolutely continuous and $X(t+h_j)\not\in   \mathbf{X}(T_t+h_j)$ for all $j$ a.s. if the joint probability density of $(X(t+h_j),  \mathbf{X}(T_t+h_j))$ exists, it is reasonable to assume that
 $$
 \mathbb{P}\left( X(t+h_j)= g( \lambda, \mathbf{X}(T_t+h_j)) \right)=0,
 $$
 $$
 \mathbb{P}\left( Y_j=  F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j))) \right)=0, \quad j=1,\ldots, N.
 $$
$$
 \mathbb{P}\left( F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_i))=  F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j))) \right)=0, \quad i, j=1,\ldots, N.
 $$

Then the subgradients $\nabla^*Q_j(\lambda)$ of \eqref{eq:Q_unconstraint} and\eqref{eq:Q1} write with probability one as
  \begin{align*}
    \nabla^*Q_j(\lambda)&= \Bigg[2\mathbbm{1}\{X(t+h_j)<g( \lambda, \mathbf{X}(T_t+h_j))  \} -1\Bigg]   p_{\widehat \theta}(g( \lambda, \mathbf{X}(T_t+h_j)) )  \nabla^* g( \lambda, \mathbf{X}(T_t+h_j)),
\end{align*}
 \begin{align*}
    \nabla^*Q_j(\lambda)&= \Bigg[\mathbbm{1}\{X(t+h_j)<g( \lambda, \mathbf{X}(T_t+h_j))  \}  + \frac{\gamma}{N}  +   \frac{2\gamma}{N} \sum_{i=1}^{j-1} \mathbbm{1}\{F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_i))< F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))\}    \Bigg. \\
    &
    -\Bigg.2\gamma F_{\widehat \theta}( g( \lambda, \mathbf{X}(T_t+h_j)))  )\Bigg]   p_{\widehat \theta}(g( \lambda, \mathbf{X}(T_t+h_j)) )  \nabla^* g( \lambda, \mathbf{X}(T_t+h_j))\\
    & +   \frac{2 \gamma}{N} \sum_{i=1}^{j-1} \mathbbm{1}\{F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_i))> F_{\widehat \theta} ( g( \lambda, \mathbf{X}(T_t+h_j)))\}  p_{\widehat \theta}(g( \lambda, \mathbf{X}(T_t+h_i)) )  \nabla^* g( \lambda, \mathbf{X}(T_t+h_i)),
\end{align*}
respectively. The subgradient $\nabla^*Q_j(\lambda)$ of \eqref{eq:Q} can be written as
\begin{align*}
    \nabla^*Q_j(\lambda)&= \Bigg[\mathbbm{1}\{X(t+h_j)<g( \lambda, \mathbf{X}(T_t+h_j))  \}  -\gamma  \mathbbm{1}\{g( \lambda, \mathbf{Y}(T_t+h_j))<  g( \lambda, \mathbf{X}(T_t+h_j))\}   \Bigg. \\
    &
    +\Bigg.2\gamma F_{\widehat \theta}( g( \lambda, \mathbf{X}(T_t+h_j)))  )\Bigg]   p_{\widehat \theta}(g( \lambda, \mathbf{X}(T_t+h_j)) )  \nabla^* g( \lambda, \mathbf{X}(T_t+h_j))\\
    &
    -\gamma  \mathbbm{1}\{g( \lambda, \mathbf{Y}(T_t+h_j))\geq  g( \lambda, \mathbf{X}(T_t+h_j))\}   p_{\widehat \theta}(g( \lambda, \mathbf{Y}(T_t+h_j)) )  \nabla^* g( \lambda, \mathbf{Y}(T_t+h_j)),
\end{align*}
where $\mathbf{Y}$ is an independent copy of $\mathbf{X}.$

Here  is  e.g.  $    \nabla^* g( \lambda, \mathbf{X}(T_t+h_j))=   \mathbf{X}(T_t+h_j)$ for infinitely divisible $X$ and
 $$    \nabla^* g( \lambda, \mathbf{X}(T_t+h_j))=  \left(   \mathbf{X}(t_i+h_j) \mathbbm{1}\left\{ \lambda_i X(t_i+h_j)=  \max_{k=1,\ldots, n}\lambda_k X(t_k+h_j)  \right\} , \quad i=1,\ldots, n \right)$$
 for max--stable $X$.

Now use the classical (batch) subgradient descent \cite{Kiwiel01} with  e.g. $\lambda_0\in\Lambda$,
\begin{equation}\label{eq:BatchGr}
\lambda_{l+1}=\Pi_\Lambda \left[ \lambda_l-\frac{\eta_l}{N} \sum_{j=1}^N  \nabla^*Q_j(\lambda_l)\right] , \quad l\in \mathbb N,\end{equation}
where $\Pi_\Lambda[\cdot]$ is the metric projection onto $\Lambda$ and $\eta_l>0$ is a step length factor which has to be tuned numerically.   The iterations stop at some $l^*$ whenever $| \lambda_{l^*+1}-\lambda_{l^*} |<\delta$ for some small threshold value $\delta>0$ yielding $\widehat \lambda=\lambda_{l^*}$.
In order to avoid costly computations at each step, a stochastic (or online) subgradient descent \cite{Bottou98} may be performed instead. Here, at each step
$ l\in \mathbb N,$ we do
\begin{equation}\label{eq:OnlineGr}
\lambda_{l+1}=\Pi_\Lambda \left[ \lambda_l-{\eta_l}   \nabla^*Q_j(\lambda_l)\right] , \end{equation}
where $j$ is chosen at random uniformly from  $\{1,\ldots, N\}$.
For the sequence $ \{  \eta_l\}$, we may require
$$ \sum\limits_{l=1}^\infty \eta_l=\infty, \quad \sum\limits_{l=1}^\infty \eta_l^2<\infty,
$$
for instance, $\eta_l=l^{-1}$.
 In addition, the {\it Polyak-Ruppert averaging} can be used after a burn--in period of length $l_0$: the resulting weight vector
$$
\widehat \lambda= \frac{1}{l-l_0} \sum\limits_{l=l_0}^{l^*-1} \lambda_l.
$$
Alternatively, we may set $\widehat \lambda$ to be equal to  the value of $ \lambda_l$ with the smallest target functional $\Phi(\lambda_l)$.
\begin{remark} \label{Rem:6} In general, we assume  the weight space $\Lambda$ to be a convex cone within $\mathbb R^n$. A more accurate choice of $\Lambda$ should reflect the constraints onto the support of the distribution of $X(t)$.   For instance, if $X(t)\ge 0$ a.s. we may take $\Lambda=\mathbb R_+^n$. However, for practical reasons of avoiding back projection $\Pi_\Lambda$ onto $\Lambda$ at each iteration step \eqref{eq:BatchGr} or \eqref{eq:OnlineGr} of subgradient descent methods,  it is better to modify the predictor $\widehat X_\lambda$ and make $g( \lambda, \cdot)$ be dependent on $\lambda^2=(\lambda^2(1),\ldots,\lambda^2(n))^\top$  instead of $\lambda=(\lambda(1),\ldots,\lambda(n))^\top$. Doing so, the formulas for the subgradient  $ \nabla^* g( \lambda, \mathbf{X}(T_t+h_j))$ have to be modified accordingly.
\end{remark}

\begin{remark} \label{Rem:7}  The advantage of  optimization formulation   \eqref{eq:Q1} in comparison with \eqref{eq:Q} is that a bootstrap step (generating instances $Y_j$ and thus increasing the variance of the forecast) is not needed for the evaluation of the subgradient $ \nabla^*Q_j(\lambda)$. However, there is a fee to pay: a more slow calculation of $ \nabla^*Q_j(\lambda)$ due to the sum inside.
\end{remark}

It is worth mentioning that many existing optimization routines (e.g. those built in R or Mathlab) can be used to minimize the functional \eqref{eq:EmpMoment} in lieu of \eqref{eq:BatchGr} or \eqref{eq:OnlineGr}. They sometimes work more accurately but are rather slow.
Under several additional assumptions, the  a.s. convergence of the stochastic gradient descent method \eqref{eq:OnlineGr} and the a.s. consistency of the forecast as $N\to\infty$ can be shown; however, this would blow up the length of this paper and thus will be the matter of future papers.

\section{Numerical examples}\label{sect:Num}
In this section, we apply the introduced prediction for the simulated data.
Although our method works for random fields on $W\subset R^d$, we take $d=1$ in order to simplify computations and presentation of results.

A random process $X$ is observed at points $\mathbb{T}_0=W_{o}\cap \mathbb{Z}_{h_1} \cup T_f,$ where $h_1=0.02,$ $W_{o}=[0,30-h_1],$ and $T_f$ is the forecast sample. We take $T_f=\{30.0,30.1,\ldots,30.9\}$ for the extrapolation and $T_f=\{30.0,30.5,31.0\ldots,34.5\}$ for the interpolation problems. In both cases $n=|T_f|=10.$

We predict the values $X(t)$ at locations $t\in \mathbb{Z}_{h_1}\cap[30,35],$ $t\not \in \mathbb{T}_0$ via predictor $\hat{X}_\lambda=\lambda^{\top}\mathbf{X},$ where $\lambda\in \mathbb R^n$ and $\mathbf{X}=(X(t_1),\ldots,X(t_n))^{\top},$ $t_j\in T_f,$ $j=1,\ldots,n.$

\begin{remark}
\label{num:Rem}
One can use the modified minimization functionals $\widetilde{\Phi}_1(\lambda)=\frac{1}{N}\sum_{j=1}^N \widetilde{Q}_{j,1}(\lambda)$ and $\widetilde{\Phi}_2(\lambda)=\frac{1}{N}\sum_{j=1}^N \widetilde{Q}_{j,2}(\lambda),$ where
\begin{align}
    \label{eq:num:u}\widetilde{Q}_{j,1}&=2F (X(t+h_j))\vee   F ( \lambda^{\top}, \mathbf{X}(T_t+h_j))- F (\lambda^{\top}\mathbf{X}(T_t+h_j))-F (X(t+h_j)),\\
    \label{eq:num:c}\widetilde{Q}_{j,2}&=\widetilde{Q}_{j,1}+\gamma\left[\frac{1}{3}+F^2 ( \lambda^{\top}\mathbf{X}(T_t+h_j))-F( \lambda^{\top}\mathbf{X}(T_t+h_j) \vee \lambda^{\top}\mathbf{Y}(T_t+h_j))\right],
\end{align}
where $\mathbf{Y}$ is a resampled version of $\mathbf{X}.$ Then the corresponding solutions $\hat{\lambda}_1=\arg \min \widetilde{\Phi}_1(\lambda)$ and $\hat{\lambda}_2=\arg \min \widetilde{\Phi}_2(\lambda)$ coincide with the minimizators of \eqref{eq:EmpMoment} with $Q_j$ being \eqref{eq:Q_unconstraint} and \eqref{eq:Q} respectively.
\end{remark}

\subsection{Gaussian random processes}
In the case of a Gaussian random process $X,$ the exact solution of minimization problem \eqref{const:eq2}  is given in by \cite[Theorem 3.5]{Dasetal22}.
For numerical illustrations, we take $X$ with standard normal marginal distribution and covariance function $C(t)=e^{-|t|/2},\, t\in\mathbb R.$

For each $T_f,$  we find numerical solutions $\hat{\lambda}_1(t)$ and $\hat{\lambda}_1(t),$ $t\in \mathbb{Z}_{h_1}\cap[30,35] \setminus T_f$. We compare the predicted trajectories $\hat{X}_u$ and $\hat{X}_c$ with ones obtained via \cite[Theorem 3.5]{Dasetal22} $\hat{X}_e$ and simple kriging $\hat{X}_{sk},$  see Figures \ref{T:Gauss:Fig:int} (interpolation) and \ref{T:Gauss:Fig:extr} (extrapolation).

While the prediction coefficients $\hat{\lambda}_1$ and $\hat{\lambda}_2$ are computed bases on one learning sample, the quality of prediction is evaluated on 1000 independently simulated trajectories on $[30,35]\cap \mathbb Z_{h_1}.$ We compute approximately the corresponding sample values of Gini metrics $G(X(t),\hat{X}(t))$ presented in Figures \ref{G:Gauss:Fig:int} and \ref{G:Gauss:Fig:extr}. The similarity between marginal distributions $X(t)$ and $\hat{X}(t)$  is measured as Wasserstein distance $\rho(F(X(t),F(\hat{X}(t))),$ whose values are in Figures \ref{W:Gauss:Fig:int} and \ref{W:Gauss:Fig:extr}.

\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Traj-Gauss-Inte-2022-06-29.PDF}
    \caption{Interpolation of a Gaussian random process $X$. True trajectory $X(t)$ (black), predicted trajectories $\hat{X}_u,$ (red, solid), $\hat{X}_c$ (blue, solid), $\hat{X}_{sk}$ (red, dashed), $\hat{X}_{e}$ (blue, dashed).}
    \label{T:Gauss:Fig:int}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Gini-Gauss-Inte-2022-06-29.PDF}
    \caption{Interpolation of a Gaussian random process $X$. Gini metric for predictors $\hat{X}_u,$ (red, solid), $\hat{X}_c$ (blue, solid), $\hat{X}_{sk}$ (red, dashed), $\hat{X}_{e}$ (blue, dashed).}
    \label{G:Gauss:Fig:int}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{WD-Gauss-Inte-2022-06-29.PDF}
    \caption{Interpolation of a Gaussian random process $X$. Wasserstein distance between $F(X)$ and predictors $F(\hat{X}_u),$ (red, solid), $F(\hat{X}_c)$ (blue, solid), $F(\hat{X}_{sk})$ (red, dashed), $F(\hat{X}_{e})$ (blue, dashed).}
    \label{W:Gauss:Fig:int}
\end{figure}

\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Traj-Gauss-Extra-2022-06-29.PDF}
     \caption{Extrapolation of a Gaussian random process $X$. True trajectory $X(t)$ (black), predicted trajectories $\hat{X}_u,$ (red, solid), $\hat{X}_c$ (blue, solid), $\hat{X}_{sk}$ (red, dashed), $\hat{X}_{e}$ (blue, dashed).}
    \label{T:Gauss:Fig:extr}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Gini-Gauss-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a Gaussian random process $X$. Gini metric for predictors $\hat{X}_u,$ (red, solid), $\hat{X}_c$ (blue, solid), $\hat{X}_{sk}$ (red, dashed), $\hat{X}_{e}$ (blue, dashed).}
    \label{G:Gauss:Fig:extr}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{WD-Gauss-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a Gaussian random process $X$. Wasserstein distance between $F(X)$ and predictors $F(\hat{X}_u),$ (red, solid), $F(\hat{X}_c)$ (blue, solid), $F(\hat{X}_{sk})$ (red, dashed), $F(\hat{X}_{e})$ (blue, dashed).}
    \label{W:Gauss:Fig:extr}
\end{figure}

\subsection{$\alpha$-stable moving averages}

We consider the moving average process
$X_\alpha=\{X_\alpha(t)=\sum_{x\in \mathbb Z}m(t/{h_1}-x)\xi_\alpha(x),\,t\in \mathbb Z_{h_1}\},$ where $\xi_\alpha(x)$ are independent $S_\alpha(1,\beta,0)-$random variables and $m:\mathbb Z\to\mathbb R_+$ is a kernel function such that $\|m\|_\alpha:=\left(\sum_{x\in \mathbb Z}m^\alpha(x)\right)^{1/\alpha}<\infty.$

The two cases of heaviness of the tails  are chosen:  Cauchy distribution ($\alpha=1,$ $\beta=0$) and L\'{e}vy distribution ($\alpha=0.5,$ $\beta=1$). Then $X$ is stationary with  marginal distribution $S_\alpha(\|m\|_\alpha,\beta,0)$.
 The kernel function is given by
$$m(x)=\begin{cases} e^{-0.02x}\left(1-e^{-0.02}\right)\left(1-e^{-5.02}\right)^{-1} \mathbbm{1}(x\in \mathbb Z\cap[0,251]),\quad&\alpha=1,\\
e^{-0.02x} \left(1-e^{-0.01}\right)^2\left(1-e^{-2.51}\right)^{-2}  \mathbbm{1}(x\in \mathbb Z\cap[0,251]),\quad&\alpha=0.5.
\end{cases}$$

It holds $\|m\|_\alpha=1$ and $X(t)\sim S_\alpha(1,\beta,0).$ We choose $S_{0.5}(1,1,0)$ and $S_1(1,0,0)$ because there are simple analytical formulas for their c.d.f.'s :
\begin{equation}
    F_{\alpha=0.5}(y)=\frac{1}{\sqrt{2\pi}}y^{-3/2}e^{-\frac{1}{2y}}\mathbbm{1}\{y>0\},\quad F_{\alpha=1}(y)=\frac{1}{2}+\frac{1}{\pi}\arctan(y),\quad y\in \mathbb R.
\end{equation}
Moreover, $\mathbf {E} |X(t)|=+\infty$ and $\mathbf {E} X^2(t)=+\infty$ in both cases.

For each $T_f,$ $X_{0.5},$ and $X_{1}$ we find numerical solutions $\hat{\lambda}(t),$ and $\hat{\lambda}_2(t),$  $t\in \mathbb{Z}_{h_1}\cap[30,35] \setminus T_f$ from Remark~ \ref{num:Rem}, which leads to predicted trajectories $\hat{X}_{0.5,u}, \hat{X}_{1,u}$ and $\hat{X}_{0.5,c}, \hat{X}_{1,c},$ respectively, see Figures \ref{T:Cauchy:Fig:int},\ref{T:Cauchy:Fig:extr},\ref{T:Levy:Fig:int}, and \ref{T:Levy:Fig:extr}.

Moreover, we apply two different methods of numerical solution. For $\alpha=1,$ we use stochastic subgadient descent method \eqref{eq:OnlineGr} and, for $\alpha=0.5,$ we apply Remark \ref{Rem:6} and existing optimization routine in Wolfram Mathematica.

We repeat the simulation procedure 1000 times and  compute approximately the corresponding values of Gini metrics $G(X(t),\hat{X}(t))$, see Figures \ref{G:Cauchy:Fig:int}, \ref{G:Cauchy:Fig:extr}, \ref{G:Levy:Fig:int} and \ref{G:Levy:Fig:extr}. The corresponding  Wasserstein distances $\rho(F(X(t),F(\hat{X}(t)))$  are in Figures \ref{W:Cauchy:Fig:int}, \ref{W:Cauchy:Fig:extr}, \ref{W:Levy:Fig:int} and \ref{W:Levy:Fig:extr}.

{\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Traj-Cauchy-Inte-2022-06-29.PDF}
    \caption{Interpolation of a moving average $X_{1}$ with Cauchy distributed marginals. True trajectory $X_{1}(t)$ (black), predicted trajectories $\hat{X}_{1,u},$ (red) and $\hat{X}_{1,c}$ (blue).}
    \label{T:Cauchy:Fig:int}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Gini-Cauchy-Inte-2022-06-29.PDF}
    \caption{Interpolation of a moving average $X_{1}$ with Cauchy distributed marginals. Gini metric for predictors $\hat{X}_{1,u},$ (red) and $\hat{X}_{1,c}$ (blue).}
    \label{G:Cauchy:Fig:int}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{WD-Cauchy-Inte-2022-06-29.PDF}
    \caption{Interpolation of a moving average $X_{1}$ with Cauchy distributed marginals. Wasserstein distance between $F(X_{1})$ and predictors $F(\hat{X}_{1,u}),$ (red), $F(\hat{X}_{1,c})$ (blue).}
    \label{W:Cauchy:Fig:int}
\end{figure}
}

{
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Traj-Cauchy-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a moving average $X_{1}$ with Cauchy distributed marginals. True trajectory $X_{1}(t)$ (black), predicted trajectories $\hat{X}_{1,u},$ (red) and $\hat{X}_{1,c}$ (blue).}
    \label{T:Cauchy:Fig:extr}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Gini-Cauchy-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a moving average $X_{1}$ with Cauchy distributed marginals. Gini metric for predictors $\hat{X}_{1,u},$ (red) and $\hat{X}_{1,c}$ (blue).}
    \label{G:Cauchy:Fig:extr}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{WD-Cauchy-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a moving average $X_{1}$ with Cauchy distributed marginals. Wasserstein distance between $F(X_{1})$ and predictors $F(\hat{X}_{1,u}),$ (red), $F(\hat{X}_{1,c})$ (blue).}
    \label{W:Cauchy:Fig:extr}
\end{figure}
}

{\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Traj-Levy-Inte-2022-06-29.PDF}
    \caption{Interpolation of a moving average $X_{0.5}$ with L\'{e}vy distributed marginals. True trajectory $X_{0.5}(t)$ (black), predicted trajectories $\hat{X}_{0.5,u},$ (red) and $\hat{X}_{0.5,c}$ (blue).}
    \label{T:Levy:Fig:int}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Gini-Levy-Inte-2022-06-29.PDF}
    \caption{Interpolation of a moving average $X_{0.5}$ with L\'{e}vy distributed marginals. Gini metric for predictors $\hat{X}_{0.5,u},$ (red) and $\hat{X}_{0.5,c}$ (blue).}
    \label{G:Levy:Fig:int}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{WD-Levy-Inte-2022-06-29.PDF}
    \caption{Interpolation of a moving average $X_{0.5}$ with L\'{e}vy distributed marginals. Wasserstein distance between $F(X_{0.5})$ and predictors $F(\hat{X}_{0.5,u}),$ (red), $F(\hat{X}_{0.5,c})$ (blue).}
    \label{W:Levy:Fig:int}
\end{figure}
}

{
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Traj-Levy-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a moving average $X_{0.5}$ with L\'{e}vy distributed marginals. True trajectory $X_{0.5}(t)$ (black), predicted trajectories $\hat{X}_{0.5,u},$ (red) and $\hat{X}_{0.5,c}$ (blue).}
    \label{T:Levy:Fig:extr}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{Gini-Levy-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a moving average $X_{0.5}$ with L\'{e}vy distributed marginals. Gini metric for predictors $\hat{X}_{0.5,u},$ (red) and $\hat{X}_{0.5,c}$ (blue).}
    \label{G:Levy:Fig:extr}
\end{figure}
\begin{figure}[p]
    \centering
    \includegraphics[width=0.8\linewidth]{WD-Levy-Extra-2022-06-29.PDF}
    \caption{Extrapolation of a moving average $X_{0.5}$ with L\'{e}vy distributed marginals. Wasserstein distance between $F(X_{0.5})$ and predictors $F(\hat{X}_{0.5,u}),$ (red), $F(\hat{X}_{0.5,c})$ (blue).}
    \label{W:Levy:Fig:extr}
\end{figure}
}

\subsection{Autoregressive stationary process}
A random process, which can be predicted nicely via linear functional is an autoregressive time series $AR(p)$ given by
$$X(t+p h)=\varphi_{1}X(t-ph)+\varphi_{2}X(t-ph+h)+\ldots+\varphi_{p}X(t-h)+\xi_t,$$
where $\sigma_{t},t\in \mathbb Z$ are independent random variables and $\varphi_k,k=1,\ldots,p$ are regression coefficients.

We examine our method on a simulated trajectory of $X(t),t\in \mathbb{T}_0\cup T_f$ with $T_f=\{30.0,30.1,30.2\}$ and $n=p=3.$ We take $\varphi_1=0.1,$ $\varphi_2=0.25,$ $\varphi_1=0.5$ in order to have a stationary $AR(p)$ process. Indeed, it is not hard to verify that the roots of the equation $1=\sum _{i=1}^{p}\varphi_{i}z^{i}$ lie outside the unit circle.
We take also $\xi_t$ having a standard Student t-distribution  with the degree of freedom $\nu=0.8,$ which is infinitely divisible with $\mathbf {E} |\xi_t|=\infty.$ However, we do not know the exact marginal distribution of $X,$ and it is impossible to apply the Gini metric. Therefore, we use the  excursion metric $E_F$ with c.d.f. $F=F_{\hat{\theta}}$ being close to the true marginal distribution of $X$ and having the same support. We choose $F_\theta,$ $\theta=(\mu,\sigma,\nu)$ from parametric family of Student t-distributions $ST(\mu,\sigma,\nu).$ Based on a simulated trajectory of $X,$ we find that  $F_{\hat{\theta}}$ with $\hat{\theta}=(0,10,0.7)$ best approaches the marginals of $X$.

In this example, we do not extrapolate the trajectories of $X$ on a wide interval, but we study the performance of the minimization algorithm. That is why the prediction coefficients $\lambda(t)=(\lambda_1(t),\lambda_2(t),\lambda_3(t))$ are computed at points $t\in\{30.3,30.4,30.5,30.6\}$. We do this for minimization problems \eqref{eq:num:c} and \eqref{eq:num:u}, via stochastic gradient descent and via standard minimization methods implemented in R language, resulting $\hat{\lambda}_k^g$ and $\hat{\lambda}_k^r,$ $k=1,2$ respectively. The results are given in Tables \ref{TB1} (unconstrained) and \ref{TB2} (constrained).

\begin{table}[ht]
\footnotesize
\centering
\begin{tabular}{cccccc}
$t$& $\hat{\lambda}_1^g(t)$ & $\widetilde{\Phi}_1(\hat{\lambda}_1^g(t))$ & $\hat{\lambda}_1^r(t)$& $\widetilde{\Phi}_1(\hat{\lambda}_1^r(t))$& $\|\hat{\lambda}_1^g(t)-\hat{\lambda}_1^r(t)\|_2$\\
\hline
  30.3 & (0.10490,\, 0.24573,\, 0.49832) & 0.04423 & (0.10832,\, 0.24299,\, 0.49852)& 0.04423 & 0.00438 \\
  30.4 & (0.05293,\, 0.22722,\, 0.49585) & 0.06358 & (0.05914,\, 0.22262,\, 0.49288) & 0.06358 & 0.00827 \\
  30.5 & (0.04532,\, 0.19385,\, 0.46395) & 0.08082 & (0.06593,\, 0.18834,\, 0.44610) & 0.08076 & 0.02782 \\
  30.6 & (0.15467,\, 0.19230,\, 0.28920) & 0.09702 & (0.05452,\, 0.16589,\, 0.41035) & 0.09579 & 0.15938 \\
  $\varphi$ & (0.10000,\, 0.25000,\, 0.50000) & &   (0.10000,\, 0.25000,\, 0.50000) & & \\
   \hline
\end{tabular}
\caption{AR(3) stationary process. Extrapolation coefficients $\hat{\lambda}_1^g$ and $\hat{\lambda}_1^r$ for unconstrained minimization.}
\label{TB1}
\end{table}

\begin{table}[ht]
\centering
\footnotesize
\begin{tabular}{cccccc}
$t$& $\hat{\lambda}^g_2(t)$ & $\widetilde{\Phi}_2(\hat{\lambda}_2^g(t))$ & $\hat{\lambda}_2^r(t)$& $\widetilde{\Phi}_2(\hat{\lambda}^r_2(t))$& $\|\hat{\lambda}_2^g(t)-\hat{\lambda}_2^r(t)\|_2$\\
\hline
  30.3 & (0.12154,\, 0.23066,\, 0.48210) & 0.04784 & (0.12081,\, 0.24737,\, 0.47881) & 0.04770 & 0.01705 \\
  30.4 & (0.05638,\, 0.23474,\, 0.50164) & 0.05212 & (0.07559,\, 0.22932,\, 0.48422) & 0.05207 & 0.02650 \\
  30.5 & (0.33977,\, 0.26291,\, 0.39731) & 0.08588 & (0.10881,\, 0.18561,\, 0.43390) & 0.07623 & 0.24629 \\
  30.6 & (-0.04932,\, 0.21927,\, 0.53541) & 0.08633 & (0.08251,\, 0.17675,\, 0.42702) & 0.08513 & 0.17588 \\
   $\varphi$ & (0.10000,\, 0.25000,\, 0.50000) & &   (0.10000,\, 0.25000,\, 0.50000) & & \\
   \hline
\end{tabular}
\caption{AR(3) stationary process. Extrapolation coefficients $\hat{\lambda}_2^g$ and $\hat{\lambda}_2^r$ for constrained minimization.}
\label{TB2}
\end{table}

\bibliographystyle{abbrv}