\section{Introduction}
The L\'{e}vy process is a fundamental tool for the study of continuous time stochastic phenomena \cite{Bertoin_1997,Barndorff-NielsenMikoschResnick2001,applebaum_2009}. The most familiar example is of course Brownian motion, but it is well known that the Gaussian assumption is inadequate for modelling of real-world phenomena, which are often more heavy-tailed than the Gaussian. Applications are found in areas such as financial modelling, \cite{Mandelbrot1963,Fama1965a,Cont:2003} communications \cite{Azzaoui2010,Fahs2012, FreitasEganClavierEtAl2017,LiebeherrBurchardCiucu2012,ShevlyakovKim2006, WarrenThomas1991},
signal processing \cite{Nikias1995}, image analysis \cite{Achim2001, Achim2006}
and audio processing \cite{Godsill_Rayner_1996,Lombardi2006}.
Non-Gaussian heavy-tailed effects are also important in the climatological sciences
\cite{Katz1992,Katz2002}, in the medical sciences
\cite{ChenWangMcKeown2010} and for the understanding of sparse modelling/Compressive Sensing \cite{UnserTaftiAminiEtAl2014, UnserTaftiSun2014, Unser2014, AminiUnser2014, CarrilloRamirezArceEtAl2016,Lopes2016, ZhouYu2017, Tzagkarakis2009, AchimBuxtonTzagkarakisEtAl2010}.

In this paper we study a very general class of non-Gaussian L\'{e}vy processes, the Generalised Inverse Gaussian process \cite{Barndorff-NielsenMikoschResnick2001,Eberlein2003}, a process which can capture various degrees of heavy-tailed behaviour, including the gamma process, the inverse Gaussian and the reciprocal-gamma process, as well as processes that lie somewhere `in between' these edge cases \cite{Eberlein2003}. Our work also enables direct simulation of the mean-variance mixtures of GIG processes, leading to the Generalised Hyperbolic (GH) processes \cite{Eberlein_2001}. Important sub-classes of the GH process include the asymmetric Student-t process which has previously been intractable for simulation and inference to our knowledge (this class is entirely distinct from the Student-t processes developed in the Machine Learning literature \cite{shah_2014}).

Rosinski \cite{Rosinski_2001} presents a generalised shot-noise representation of non-Gaussian L\'{e}vy processes, and it is this approach that we develop here for the GIG and GH processes. Our previous work using the shot noise representation has focussed on stable law processes and their applications in engineering, see \cite{Lemke_Godsill_2015,Riabiz2020,Godsill_Riabiz_Kont_2019} and references therein. There have been relevant developments in various special cases over recent years, including
\cite{B_N_1997} who present the theory of normal-inverse Gaussian (NIG) processes,
\cite{Rydberg_1997} who present approximate sampling methods for the  NIG  case, while
\cite{B_N_Shephard_2001} give applications of series based methods for non-Gaussian Ornstein-Uhlenbeck (OU) processes.
\cite{Zhang_2011} provided rejection sampling and series based simulations for the  GIG OU process using the concepts of \cite{Rosinski_2001}, but these are applied to a different Background driving L\'{e}vy process (BDLP) than our work and indeed require evaluation of a generally intractable integral involving Bessel Functions. In addition \cite{Zhang_2011a,Qu_etal_2020} have provided exact simulation methods for the related class of tempered stable (TS) processes.

The distribution of the GIG L\'{e}vy process at $t=1$, the GIG distribution, possesses
a three parameter probability density function defined for positive real random variables as follows \cite{Eberlein2003}:

\begin{equation}
    f_{GIG}(x) = \left( \frac{\gamma}{\delta} \right)^{\lambda} \frac{1}{2 K_{\lambda}(\delta \gamma)} x^{\lambda-1} e^{-\frac{1}{2} \left( \delta^2 x^{-1} + \gamma^2x \right)} \mathcal{I}_{x > 0}
\end{equation}

\noindent where $\lambda \in \mathbb{R}$, $K_{\lambda}(\cdot)$ is the modified Bessel function of the second kind and $\cal I$ is the indicator function. The support of parameters $\gamma$ and $\delta$ depend on the sign of $\lambda$ as discussed in \cite{Eberlein2003}. It is shown in \cite{Barndorffnielsen1977InfiniteDO} that the GIG distribution is infinitely divisible and hence can be the distribution of a L\'{e}vy process at time $t=1$. Furthermore, particular values of the parameters lead to special cases of the GIG distribution such as the Inverse Gaussian $(\lambda = -1/2)$, Gamma ($\delta=0$, $\lambda>0$) and the Reciprocal-Gamma ($\gamma = 0$, $\lambda<0$) distributions, and in these limit cases the normalising constant is adjusted to the normalising constant of these well known distributions.

The principal contribution of this paper is to provide a comprehensive suite of methods for simulation of GIG processes, without the need for evaluation of intractable integrals, beyond pointwise evaluation of the relevant Bessel functions.  An auxiliary variables approach transforms the univariate GIG point process into a bivariate point process having the GIG process as its marginal by construction, and requiring no explicit evaluation of integrals. We derive tractable dominating measures for the augmented GIG L\'{e}vy density, hence leading to a random thinning methodology for generation of jumps of the underlying marginal  GIG process. The whole procedure is carried out through generation of familiar random variables (from the gamma family) and point processes (both gamma and tempered stable). In addition we are able to bound the average acceptance rates of the random variate generation and also to provide upper and lower bounds on the  GIG L\'{e}vy density and the corresponding Jaeger integral. Finally the whole methodology is made accessible to practitioners through the publication of Matlab and Python code to implement the GIG sampling schemes.

Section 2 presents the necessary preliminaries for simulation of L\'{e}vy processes and their corresponding point processes, using a generalised shot-noise approach. Section 3 gives the specific forms for the GIG L\'{e}vy density and derives bounds on these densities, as well as a generic thinning method for tractable sampling of the underlying point processes, and presents in detail the simulation method for two different parameter ranges of the process. Section 4 presents example simulations, compared with exact simulations of the GIG random variable and finally Section 5 discusses the application of GIG processes in simulation and inference for more general processes including the Generalised Hyperbolic Process.

\section{Shot noise representations, L\'{e}vy densities and thinning}

In this section we present the required preliminaries about L\'{e}vy processes and shot-noise simulation of those processes, using the framework of \cite{Rosinski_2001}. The characteristic function for a general non-negative-valued  L\'{e}vy process having no drift or Brownian motion part is given by \cite{Kallenberg_2002}, Corollary 15.8, as:

\begin{equation}
    E \left[ \exp(iuW(t)) \right] = \exp t \left[\int_{(0,\infty)} (e^{iuw} -1 - iuw{\cal I}_{|w|\leq 1})Q(dw) \right]
\end{equation}

\noindent where $Q$ is a L\'{e}vy measure on $(0,\infty)$. By the L\'{e}vy-Ito integral representation, we may express $W(t)$ directly as:

\begin{equation}
    W(t) = \int_{|w|\leq 1} w [(N([0, t], dw) - tQ(dw)] + \int_{|w|>1} wN([0, t], dw) \label{eq:Levy_direct}
\end{equation}

\noindent Here $N$ is a bivariate point process having mean measure $Lebesgue \times Q$ on $[0,T]\times (0,\infty)$, which can be conveniently expressed as:

\[
N=\sum_{i=1}^\infty \delta_{V_i,J_i}
\]

\noindent where  $\{V_i\in[0,T]\}$ are i.i.d. uniform random variables which give the times of arrival of jumps, $\{J_i\}$ are the size of the jumps and $\delta_{V,J}$ is the Dirac measure centered at time $V$ and jump size $J$.

Substituting $N$ directly into (\ref{eq:Levy_direct}) leads to

\[
W(t)=\sum_{i=1}^\infty J_i{\cal I}_{V_i\leq t}-tc_i,\,\,\,as
\]

\noindent where $c_i$ are compensator terms, which may be taken as zero for the cases considered in this paper. The almost sure convergence of this series to $\{W(t)\}$ is proven in \cite{Rosinski_2001}.

In principle it may be possible to simulate directly from the point process $N$, but in practice the potentially infinite number of jumps in any finite time interval make this impossible. If the jumps can be ordered by size, however, it may be possible to simulate all of the significant jumps and ignore or approximate the residual error from omitting the smallest jumps. It turns out that this can be done in a very convenient way, using the well known approach of \cite{Ferguson_Klass,Rosinski_2001}. The starting point is the simulation of the epochs of a unit rate Poisson process $\{\Gamma_i\}_{i\geq 1}$. The intensity function of this process is of course $\lambda_\Gamma=1$, and the resulting realisations contain almost surely an infinite number of terms. We know how to simulate an arbitrarily large number of ordered terms from this process by repeatedly generating exponential random variables and calculating the cumulative sum of these to obtain an ordered $\Gamma_i$ sequence. Now take a deterministic, non-increasing function $h(\Gamma)$ of the epochs to obtain a transformed point process:

\[
    N'=\sum_{i=1}^\infty \delta_{V_i,h(\Gamma_i)}
\]

\noindent then we would like this process to be equivalent to the original point process $N$. Suppose that the L\'{e}vy measure for $N$ is $Q(\cdot)$, defined for positive support $J\in (0,\infty )$ then we can determine the appropriate function $h(\cdot)$ to obtain the correct point process $N$. This is straightforward to achieve. Note that for all $\gamma\in [0,\infty)$,

\[
E \left[ \# \{ J_i; J_i \geq j \} \right] = h^{-1}(j)
\]

\noindent But for the point process $N$ it is  required that $E[\#\{J_i; J_i\geq j\}] = Q ([j,\infty))= Q^+(j)$ where $Q^+(j)$ is defined as the upper tail probability of the L\'{e}vy measure. So, finally,

\[
h(\gamma)={Q^+}^{-1}(\gamma)
\]

\noindent where ${Q^+}^{-1}(\cdot)$ is the inverse of the upper tail probability of the L\'{e}vy measure. This is a well known result, proposed for simulation of L\'{e}vy processes by many authors including \cite{Ferguson_Klass,Rosinski_2001}. Thus, to generate points directly from $N$ by this method it is necessary to be able to compute ${Q^+}^{-1}(\cdot)$ explicitly, either in closed form or by numerical approximations. In some cases where this is not possible it may nevertheless be possible to simulate directly from a bounding process $N_0$ with $dQ_0(x)/dQ(x)\geq 1$ $\forall x\in(0,\infty)$ and then to thin with probability $dQ(x)/dQ_0(x)$ in order to obtain samples from $N$ \cite{Lewis_Shedler_1979}. In all cases considered here the L\'{e}vy measure possesses a density function, which we also denote by $Q(x)$ (using the minor abuse of notation `$dQ(x)=Q(x)dx$') and the required bounding condition is then $Q_0(x)\geq Q(x)$, with associated thinning probability $Q(x)/Q_0(x)$.

Simulation examples required for this paper are the tempered stable (TS) and gamma processes, which may be sampled as follows:

\subsection{Tempered stable point process}
\label{TS_sample}
In the tempered stable (TS) case the L\'{e}vy density is, for $\alpha\in(0,2)$ \cite{Shephard_BN_2001} (see also \cite{Rosinski_2007})

\[
    Q(x) = Cx^{-1-\alpha} e^{-\beta x}
\]

\noindent The tail probability may be calculated in terms of gamma functions, but is not easily inverted. Instead it may be split into a positive $\alpha$-stable process \cite{Samorodnitsky_Taqqu_1994} with density $Q_0(x)=Cx^{-1-\alpha}$ and a tempering function $e^{-\beta x}$. The stable law process has tail mass $Q_0^+(x)=\frac{C}{\alpha}x^{-\alpha}$ and hence $h(\gamma)=Q_0^+(\gamma)^{-1}=\left(\frac{\alpha\gamma}{C}\right)^{-1/\alpha}$ \cite{Samorodnitsky_Taqqu_1994}. Having simulated the point process with rate function $Q_0$, points are individually selected (thinned) with probability $e^{-\beta x}$, otherwise deleted.

\subsection{Gamma process}
\label{Gamma_process}
The  L\'{e}vy density for the Gamma process is:
\[
Q(x)={C}{x^{-1}}e^{-\beta x}
\]
\cite{Rosinski_2001} suggests four possible sampling schemes for this process. The tail probability is the Exponential integral, which would require numerical inversion, see \cite{Wolpert_Ickstadt_1998}. Instead we adopt the thinning (also called the `rejection method' in \cite{Rosinski_2001}) version of this in which a dominating point process is chosen as $Q_0(x)=\frac{C}{x}(1+\beta x)^{-1}$. The tail probability for this is $Q_0^+(x)=C\log\left(\beta^{-1} x^{-1}+1\right)$ and hence $h(\gamma) = \frac{1}{\beta \left( \exp(\gamma / C) - 1 \right)}$. Points are then  thinned with probability $(1+\beta x) \exp(-\beta x) \leq 1$. As reported in \cite{Rosinski_2001}, this thinning method is highly effective, with very few point rejections observed.

\section{Simulation from the Generalised inverse Gaussian L\'{e}vy process}

In this section the GIG L\'{e}vy density is presented and a general scheme is presented that will enable simulation from the GIG process.

The L\'{e}vy density for the Generalised inverse Gaussian (GIG) process is given by \cite{Eberlein2003}:

\[
\frac{e^{-x\gamma^2/2}}{x}\left[\int_0^\infty\frac{e^{-xy}}{\pi^2y|H_{|\lambda|}(\delta\sqrt{2y})|^2}dy+\text{max}(0,\lambda)\right],\,\,x>0
\]
where
$H_{\lambda}(z)=J_{\lambda}(z)+iY_{\lambda}(z)$ is the Bessel function of the third kind, also known as the Hankel function, $J_{\lambda}(z)$ is the Bessel function of the first kind, and  $Y_{\lambda}(z)$ is the  Bessel function of the second kind.

The GIG L\'{e}vy density comprises two terms: the initial integral, which we denote $Q_{GIG}(x)$:
\[
Q_{GIG}(x)=\frac{e^{-x\gamma^2/2}}{x}\int_0^\infty\frac{e^{-xy}}{\pi^2y|H_{|\lambda|}(\delta\sqrt{2y})|^2}dy,\,\,x>0
\]
added to a second term, present only for $\lambda>0$:
\[
\frac{e^{-x\gamma^2/2}}{x}\text{max}(0,\lambda),\,\,x>0
\]
This second term is a Gamma process that may be straightforwardly simulated using the standard methods of \ref{Gamma_process} and added to the simulation of points from the first term $Q_{GIG}(x)$. Hence we will neglect this second term for now.
 It will be convenient to rewrite $Q_{GIG}(x)$ using the substitution $z=\delta\sqrt{2y}$ as:

\[
Q_{GIG}(x)=\frac{e^{-x\gamma^2/2}}{x}\int_0^\infty\frac{e^{-xy}}{\pi^2y|H_{|\lambda|}(\delta\sqrt{2y})|^2}dy=\frac{2 e^{-x\gamma^2/2}}{\pi^2x}\int_0^\infty\frac{e^{-\frac{z^2x}{2\delta^2}}}{z|H_{|\lambda|}(z)|^2}dz
\]

Note that this integral is known elsewhere as the Jaeger Integral, which finds application in diffusive transport \cite{Freitas2018}. Beyond our direct interest in GIG processes, there is significant interest in approximating these integrals accurately, and our bounding approach is likely to provide accurate bounds and approximations which may be compared with those proposed in \cite{Freitas2018}.

Throughout this paper we will consider only positive and real-valued variables $x$, $y$ and $z$, as the complex versions of these functions are not required in the present context.

Our general scheme is to consider the following bivariate point process on $(0,\infty)\times (0,\infty)$:
\begin{equation}
Q_{GIG}(x,z)=\frac{2 e^{-x\gamma^2/2}}{\pi^2x}\frac{e^{-\frac{z^2x}{2\delta^2}}}{z|H_{|\lambda|}(z)|^2}\label{Q_bivariate}
\end{equation}
which has by construction the GIG process as its marginal:
\[
Q_{GIG}(x)=\int_{0}^\infty Q_{GIG}(x,z)dz
\]
We propose to simulate points directly from this bivariate process, hence avoiding any direct evaluation of the Jaeger Integral. Since $Q_{GIG}(x,z)$ is intractable itself for simulation, dominating processes $Q^0_{GIG}(x,z)\geq Q_{GIG}(x,z)$ are constructed and points sampled from $Q^0_{GIG}(x,z)$ are thinned with probability $Q_{GIG}(x,z)/Q^0_{GIG}(x,z)$. Thus a significant part of our approach is in constructing suitable dominating functions that are tractable for simulation, and this is achieved by studying the properties of the Jaeger Integral.

The first set of bounds is obtained from the basic properties of the Hankel function \cite{Watson1944} and will lead in particular to a simulation algorithm for the case $|\lambda|\geq 0.5$:

\begin{theorem}
\label{Lemma_1}
For $|\lambda|\geq 0.5$, the following bound applies:
\begin{equation}
Q_{GIG}(x,z)\leq \frac{ e^{-x\gamma^2/2}}{\pi x}{e^{-\frac{z^2x}{2\delta^2}}}\label{Q_GIG_bound}
\end{equation}
and, for $|\lambda|\leq 0.5$,
\begin{equation}
Q_{GIG}(x,z)\geq\frac{ e^{-x\gamma^2/2}}{\pi x}{e^{-\frac{z^2x}{2\delta^2}}}\label{Q_GIG_bound_l_0_5}
\end{equation}
\end{theorem}
\begin{proof}
We have an asymptotic expansion for $|H_{\nu}(z)|^2$ (\cite{Watson1944}, Section 13.75):

\[
    |H_\nu(z)|^2 \sim \frac{2}{\pi z}\sum_{m=0}^\infty\{1\times 3\,...\times\,(2m-1)\}\frac{(v,m)}{2^mz^{2m}}
\]

\noindent where $(\nu, m)$ is defined as (\cite{Watson1944}, Section 13.75) sec. 7.2 p. 198):

\[
(v,m)=\frac{\Gamma(\nu+m+1/2)}{m!\Gamma(\nu-m+1/2)}=\frac{\{4\nu^2-1^2\} \,\{4\nu^2-3^2\}\,...\{4\nu^2-(2m-1)^2\}}{2^{2m}m!}
\]

\noindent so that in particular we have:

\[
(\nu,0)=1\]\[(\nu,1)=\frac{(4\nu^2-1)}{4}=\begin{cases}<0,&\nu<0.5\\>0,&\nu>0.5\end{cases}
\]

\[
(\nu,2)=\frac{(4\nu^2-1)(4\nu^2-9)}{2^5}=\begin{cases}>0,&\nu<0.5\\<0,&\nu>0.5\end{cases}
\]

Moreover, when $p>\nu-1/2$, the remainder after $p$ terms `have the same sign as, and numerically less than, the $(p+1)$th term' (\cite{Watson1944}, Section 13.75). Thus,

\[
|H_\nu(z)|^2 \sim \frac{2}{\pi z}\left(1+\frac{4\nu^2-1}{8z^2}+\frac{3(4\nu^2-1)(4\nu^2-9)}{2^7z^4} +O(z^{-6})\right)
\]
Using a truncated version of the asymptotic expansion, we find the following lower bounds:
\begin{align}
|H_\nu(z)|^2\geq \begin{cases}\frac{2}{\pi z},& \nu\geq 0.5\label{H_bound}\\\frac{2}{\pi z}\left(1+\frac{(4\nu^2-1)}{8z^2}\right),&\nu<0.5\end{cases}
\end{align}
and the following upper bound for $\nu\leq 0.5$:
\begin{align}
|H_\nu(z)|^2\leq \frac{2}{\pi z}\label{H_bound_nu_l_0_5}
\end{align}
    The results follow by substitution of (\ref{H_bound}) and (\ref{H_bound_nu_l_0_5}) into $Q_{GIG}(x,z)$ (\ref{Q_bivariate}).
\end{proof}

\begin{corollary}
\label{cor_GIG_bound}
 The bound in (\ref{Q_GIG_bound}), applicable for $|\lambda|\geq 0.5$, can be rewritten as
 \begin{equation}
 \frac{ e^{-x\gamma^2/2}}{\pi x}{e^{-\frac{z^2x}{2\delta^2}}}=\frac{ {\delta\Gamma(1/2)}e^{-x\gamma^2/2}}{\sqrt{2}\pi x^{3/2}}\sqrt{{Ga}}\left(z|1/2,\frac{x}{2\delta^2}\right)\label{temp_stable}
 \end{equation}
 where $\sqrt{Ga}$ is the square-root gamma density, i.e. the density of $X^{0.5}$ when $X\sim Ga(\alpha,\beta)$, having probability density function
 \[
        \sqrt{\text{Ga}}(\alpha, \beta) = \frac{2 \beta^{\alpha}}{\Gamma(\alpha)} x^{2\alpha-1} e^{-\beta x^2}
 \]

It can be seen immediately that (\ref{temp_stable}) corresponds marginally to  a tempered stable process in $x$, and conditionally to a $\sqrt{Ga}$ density for $z$, a tractable feature that will enable sampling from the dominating bivariate point process.

Integration wrt $z$ leads to a simple upper bound on the GIG L\'{e}vy density:
\begin{equation}
\frac{ {\delta\Gamma(1/2)}e^{-x\gamma^2/2}}{\sqrt{2}\pi x^{3/2}} \label{GIG_upper}
\end{equation}
which was also obtained by \cite{Zhang_2011}, and used in a rejection sampling procedure that requires a direct evaluation of the Jaeger integral, in contrast with our approach. For $|\lambda|\leq 0.5$, a similar argument leads to a lower bound with the same formula as (\ref{GIG_upper}).
\end{corollary}

\begin{remark}
\noindent The first bound for $\nu\geq 0.5$ in (\ref{H_bound}) will be used shortly as an
envelope function for rejection sampling in the case $|\lambda|\geq 0.5$ using Corollary \ref{cor_GIG_bound}. However, the second bound in (\ref{H_bound}) for $|\lambda|<0.5$ would lead to an improper envelope function that is not integrable wrt $z$, hence we will need to develop more sophisticated bounds for the case $|\lambda|<0.5$.
\end{remark}
A second set of bounds can be stated as follows.
\begin{theorem}
\label{Lemma_2}
Choose a point $z_0\in(0,\infty)$ and compute $H_0=z_0|H_{\nu}(z_0)|^2$. This will define the corner point on a  piecewise lower or upper bound. Define now $z_1 = \left(\frac{ 2^{1-2\nu}\pi}{\Gamma^2(\nu)}\right)^{1/(1-2\nu)}$ and define the following functions:

\[
A(z)=\begin{cases}z(\Gamma(\nu)/\pi)^2\left(\frac{2}{z}\right)^{2\nu},&z < z_1 \\ \frac{2}{\pi},&z\geq z_1 \end{cases}
\]
and
\[
B(z)=\begin{cases}H_0\left(\frac{z_0}{z}\right)^{2\nu-1},&z<z_0\\H_0,&z\geq z_0\end{cases}
\]
Then, for $0<\nu\leq 0.5$,
\begin{equation}
A(z)\geq z|H_{\nu}(z)|^2\geq B(z) \label{bound_lam_leq_0_5}
\end{equation}
and for $\nu\geq 0.5$,
\begin{equation}
A(z)\leq z|H_{\nu}(z)|^2\leq B(z) \label{bound_lam_geq_0_5}
\end{equation}
with all inequalities becoming equalities when $\nu=0.5$, and both $A(z)$ bounds (left side inequalities) becoming tight at $z=0$ and $z=\infty$.
\end{theorem}
\begin{proof}
 First note that $z|H_{\nu}(z)|^2$ is an increasing function for $0 < \nu < 0.5$ and decreasing for $\nu > 0.5$  (\cite{Watson1944}, Section 13.75). When combined with the tight bound of $2/\pi$ as $z\rightarrow \infty$ (\ref{H_bound}) this leads directly to the $2/\pi$ components of $A(z)$.
 Similarly the first part of the bound $A(z)$, $z(\Gamma(\nu)/\pi)^2\left(\frac{2}{z}\right)^{2\nu}$, is justified since \cite{Freitas2018} proves that
 $z^{2\nu}|H_{\nu}(z)|^2$ is increasing when $\nu>0.5$ and decreasing when $\nu<0.5$, with a tight bound of $(\Gamma(\nu)/\pi)^2 2^{2\nu}$ as $z\rightarrow 0$.
 Thus the left hand side of the inequalities, $A(z)$, are established.

 The right hand side of the inequalities, $B(z)$, are proven by the monotonicity and sign of gradient for both $z^{2\nu}|H_{\nu}(z)|^2$ and $z|H_{\nu}(z)|^2$. We may choose an arbitrary corner point $(z_0,H_0)$ on
 $z|H_{\nu}(z)|^2$ and monotonicity of $z|H_{\nu}(z)|^2$ immediately implies that \[z|H_{\nu}(z)|^2\begin{cases}\geq H_0,&0<\nu\leq 0.5\\ \leq H_0,&\nu\geq 0.5\end{cases}\]
 Moreover, monotonicity and sign of gradient of $z^{2\nu}|H_{\nu}(z)|^2$ \cite{Freitas2018} imply that
\begin{figure}[ht]
\centering
\includegraphics[width=0.6\textwidth]{bounds_lambda=0.3.png}
\caption{Plot of Bessel function bounds, $\nu=0.3$. $z_0$ set equal to $z_1$.}
 \label{bounds_lambda_0_3}
 \includegraphics[width=0.6\textwidth]{bounds_lambda=0.8.png}
\caption{Plot of Bessel function bounds, $\nu=0.8$. $z_0$ set equal to $z_1$.}
 \label{bounds_lambda_0_8}
\end{figure}
 \[
 z^{2\nu}|H_{\nu}(z)|^2\begin{cases}\geq z^{2\nu-1}H_0,&0<\nu\leq 0.5\\ \leq z^{2\nu-1} H_0,&\nu\geq 0.5\end{cases}
 \]
 from which the bounds $B(z)$ are immediately obtained.
 \end{proof}
 \begin{remark}
 The proven bounds can be clearly visualised on a log-log plot, which highlights the asymptotic behaviour of the functions and bounds as $z\rightarrow 0 $ and $z\rightarrow \infty$, see Figs. \ref{bounds_lambda_0_3} and \ref{bounds_lambda_0_8}.
 \end{remark}
 \begin{remark}
 The choice of $z_0\in (0,\infty)$ is arbitrary. However, its value will impact the tightness of the bounding functions $B(z)$  and will hence impact the effectiveness of our subsequent sampling algorithms and integral approximations. A suitable generic choice was  found to be at the same $z$ value as the corner point of the corresponding $A(z)$  bounds, i.e. set $z_0=z_1$, as plotted in Figs. \ref{bounds_lambda_0_3} and \ref{bounds_lambda_0_8}, though further optimisation may be possible in applications.
 \end{remark}

 \begin{corollary}
 \label{cor_piecewise}
 For the case $|\lambda|<0.5$, the right hand bound in (\ref{bound_lam_leq_0_5}), $B(z)$,  can be substituted into $Q_{GIG}(x,z)$ (\ref{Q_bivariate}) and rearranged to obtain:
\begin{align}
Q_{GIG}(x,z)=
\frac{2 e^{-x\gamma^2/2}}{\pi^2x}\frac{e^{-\frac{z^2x}{2\delta^2}}}{z|H_{|\lambda|}(z)|^2}&\leq
\frac{ e^{-x\gamma^2/2}}{\pi^2x}
\begin{cases}
\frac{z^{2|\lambda|-1}e^{-\frac{z^2x}{2\delta^2}}}{H_0z_0^{2|\lambda|-1}},&0<z<z_0\\
\frac{e^{-\frac{z^2x}{\delta^2}}}{H_0},&\text{otherwise}
\end{cases}\nonumber
\\&=
\frac{ e^{-x\gamma^2/2}}{\pi^2x^{1+|\lambda|}}\frac{(2\delta^2)^{|\lambda|}\gamma(|\lambda|,z_0^2x/(2\delta^2))\frac{\Gamma(|\lambda|)\sqrt{\text{Ga}}
(z||\lambda|,x/(2\delta^2))}{\gamma(|\lambda|,z_0^2x/(2\delta^2))}}{H_0z_0^{2|\lambda|-1}}{\cal I}_{0<z<z_0}\nonumber
\\&\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,+\frac{ e^{-x\gamma^2/2}}{\pi^2x^{3/2}}\frac{(2\delta^2)^{0.5}\Gamma(0.5,z_0^2x/(2\delta^2))\frac{\Gamma(0.5)\sqrt{\text{Ga}}
(z|0.5,x/(2\delta^2))}{\Gamma(0.5,z_0^2x/(2\delta^2))}}{H_0}{\cal I}_{z\geq z_0} \label{GIG_bound_lam_le_0_5}
\end{align}
This can be split up into a two tractable point processes for simulation: a first, $N_1$, comprising a modified tempered $|\lambda|$-stable process with truncated $\sqrt{Ga}$ conditional for $z$, and a second, $N_2$, comprising a modified tempered 0.5-stable process with a further truncated $\sqrt{Ga}$ conditional for $z$. We will use this union of point processes as the dominating L\'{e}vy measure in simulation of the $|\lambda|<0.5$ case.

 \end{corollary}
  \begin{corollary}
 Integration of the bounding function (\ref{GIG_bound_lam_le_0_5}) wrt $z$ allows a more sophisticated estimate for the Jaeger integral and therefore the L\'{e}vy density for the GIG process, which is an upper bound for $|\lambda|<0.5$ and a lower bound for $|\lambda|>0.5$, following the direction of the right hand inequalities in (\ref{GIG_bound_lam_le_0_5}) and (\ref{bound_lam_geq_0_5}). A similar procedure inserting the left hand ($A(z)$) inequalities from (\ref{GIG_bound_lam_le_0_5}) and (\ref{bound_lam_geq_0_5}) yields the corresponding lower ($|\lambda|<0.5$) and upper ($|\lambda|>0.5$) bounds. Define first the  bounding functions $Q^A_{GIG}(x)$ and $Q^B_{GIG}(x)$, corresponding to the bounds $A(z)$ and $B(z)$ as follows:

\[
Q^A_{GIG}(x) =
\frac{ e^{-x\gamma^2/2}}{\pi x^{1+|\lambda|}}\frac{(2\delta^2)^{|\lambda|}\gamma(|\lambda|,z_1^2x/(2\delta^2))}{2z_1^{2|\lambda|-1}}+\frac{ e^{-x\gamma^2/2}}{\pi x^{3/2}}\frac{(2\delta^2)^{0.5}\Gamma(0.5,z_1^2x/(2\delta^2))}{2}
\]

\[
Q^B_{GIG}(x) =
\frac{ e^{-x\gamma^2/2}}{\pi^2 x^{1+|\lambda|}}\frac{(2\delta^2)^{|\lambda|}\gamma(|\lambda|,z_0^2x/(2\delta^2))}{H_0z_0^{2|\lambda|-1}}+\frac{ e^{-x\gamma^2/2}}{\pi^2x^{3/2}}\frac{(2\delta^2)^{0.5}\Gamma(0.5,z_0^2x/(2\delta^2))}{H_0}
\]
 These are obtained by substituting the bounds $A(z)$ or $B(z)$ into the expression for $Q_{GIG}(x,z)$ (\ref{Q_bivariate}) and integrating wrt $z$.

 Noting that $z_0\in (0,\infty)$ can be chosen arbitrarily, the $Q^B_{GIG}$ estimates may be improved by optimising wrt $z_0$ to either maximise ($|\lambda|< 0.5$) or minimise ($|\lambda|> 0.5$) $Q^B_{GIG}(x)$ at each point $x$.

 Then, following the direction of the  inequalities in (\ref{bound_lam_leq_0_5}) and (\ref{bound_lam_geq_0_5})
 we obtain the following estimates of $Q_{GIG}(x)$:
 \[
 Q^A_{GIG}(x)\leq Q_{GIG}(x)\leq \underset{z_0\in (0,\infty)}{\mathrm{min}}\left\{Q^B_{GIG}(x)\right\},\,\,\,\,|\lambda|\leq 0.5
 \]
\[
 Q^A_{GIG}(x)\geq Q_{GIG}(x)\geq \underset{z_0\in (0,\infty)}{\mathrm{max}}\left\{Q^B_{GIG}(x)\right\},\,\,\,\,|\lambda|\geq 0.5
 \]
 with all inequalities becoming equalities at $|\lambda|=0.5$. Optimisation to achieve the required maximum and minimum functions can be achieved by numerical search.

 Example plots are given in Fig. \ref{jaeger_integral} in which the optimised lower and upper bounds are plotted, showing very close agreement (often indistinguishable by eye) in their estimation of $Q_{GIG}(x)$, over various parameter ranges and large $x$ ranges (i.e. the bounds are visually quite tight).  Overlaid are the $x^{-3/2}$  and $x^{-(1+|\lambda|)}$ trends and also the simple approximation of (\ref{GIG_upper}).
  This simple approximation diverges significantly from our proposed bounds, particularly when $|\lambda|$ is not close to 0.5 (when $|\lambda|=0.5$ all bounds are equal to the true function $Q_{GIG}(x)$ for the Inverse Gaussian (IG) process).

\end{corollary}

\begin{figure}[ht]
\centering
\includegraphics[width=0.6\textwidth]{jaeger_integral_bounds6.jpg}
\caption{Plot of optimised upper and lower bounds on $Q_{GIG}(x)$, for various parameter settings. We use the following shorthand for the optimised $Q^B$ bounds: ${Q^B_{GIG}}^{*}(x) = \underset{z_0\in (0,\infty)}{\mathrm{max/min}}\left\{Q^B_{GIG}(x)\right\}$, where $\text{min}$ applies for $|\lambda|<0.5$ and $\text{max}$ applies for $|\lambda|\geq 0.5$ . Overlaid also are the simple approximation (\ref{H_bound}) and the trends $x^{-3/2}$ and $x^{-(1+|\lambda|)}$.}
 \label{jaeger_integral}
\end{figure}

\subsection{Simulation of GIG processes with $|\lambda|\geq 0.5$}
In this section the specific algorithm applied for simulation in the case $|\lambda|\geq 0.5$ is detailed. In this parameter range it has been found very effective to use Theorem \ref{Lemma_1} and Corollary \ref{cor_GIG_bound}, so that the dominating bivariate process is:
\[
 Q^0_{GIG}(x,z)=\frac{ e^{-x\gamma^2/2}}{\pi x}{e^{-{(z^2x)}/{(2\delta^2)}}}
\]
The thinning probability for points drawn from the dominating point process is then:
\[
\frac{Q_{GIG}(x,z)}{Q^0_{GIG}(x,z)}=\frac{2}{\pi z|H_{|\lambda|}(z)|^2}\]

The recipe for generation of points from the bivariate point process is:

\begin{enumerate}
    \item Generate a large number of points from the tempered stable process with intensity
    function, sorted large to small using a shot noise series:
    \[
    \frac{e^{-x\gamma^2/2}}{x^{3/2}}\frac{\delta\Gamma(1/2)}{\sqrt{2}\pi}
    \]
    \item For each point $x$, draw a random variate $z\sim \sqrt{\text{Ga}}(z|1/2,x/(2\delta^2))$
    \item Accept with probability
    \[
   \frac{2}{\pi z|H_{|\lambda|}(z)|^2}
    \]
\end{enumerate}

\subsubsection{Acceptance rate}

\begin{figure}[ht]
\centering
\includegraphics[width=0.6\textwidth]{bound_tightness_plot2.png}
\caption{Plot of upper and optimised lower bounds on $\rho(x)$, for various $|\lambda|>0.5$, showing that large jumps are rejected while with increasing probability small jumps are accepted. $\delta=0.1$ in all cases and bounds do not depend on $\gamma$}
 \label{rho_upper_lower_bounds}
\end{figure}

The mean acceptance probability for fixed $x$ is:

\begin{align*}
\rho(x)=E \left[ \frac{2}{\pi z|H_{|\lambda|}(z)|^2} \right] &=\int_{0}^{\infty} \sqrt{\text{Ga}}(z|1/2,x/(2\delta^2))\frac{2}{\pi z|H_{|\lambda|}(z)|^2}dz
\end{align*}

\noindent which may be bounded using Theorem \ref{Lemma_2}. For the current parameter setting ($|\lambda|>0.5$) we have $A(z) \leq z |H_{|\lambda|}(z)|^2 \leq B(z)$. Lower and upper bounds are obtained by substituting $A(z)$ and $B(z)$ and then by direct integration to give:

\begin{align}
\label{eqn:rho_lower_bound}
\frac{2}{\pi H_0} \biggl( \left(\frac{z_0^2x}{2\delta^2}\right)^{0.5-|\lambda|} \frac{\gamma(|\lambda|, z_0^2x/(2\delta^2))}{\Gamma(0.5)}& + \frac{\Gamma(0.5, z_0^2x/(2\delta^2))}{\Gamma(0.5)} \biggr)\\
&\leq \rho(x) \leq \left(\frac{z_1^2x}{2\delta^2}\right)^{0.5-|\lambda|} \frac{\gamma(|\lambda|, z_1^2x/(2\delta^2))}{\Gamma(0.5)} + \frac{\Gamma(0.5, z_1^2x/(2\delta^2))}{\Gamma(0.5)}\nonumber
\end{align}
As noted before, the corner point $z_0\in (0,\infty)$ may be chosen arbitrarily, while $z_1$ is fixed. Hence the lower bound may be optimised wrt $z_0$ at each $x$ value to obtain a tighter lower bound:
\[
\underset{z_0\in (0,\infty)}{\text{max}}\left\{\frac{2}{\pi H_0} \left( \left(\frac{z_0^2x}{2\delta^2}\right)^{0.5-|\lambda|} \frac{\gamma(|\lambda|, z_0^2x/(2\delta^2))}{\Gamma(0.5)} + \frac{\Gamma(0.5, z_0^2x/(2\delta^2))}{\Gamma(0.5)} \right)\right\}\leq \rho(x)
\]

The (numerically optimised) lower bound and (fixed) upper bound on the mean acceptance rate $\rho(x)$ are visualised  for different parameter settings in Fig. \ref{rho_upper_lower_bounds}. It can be observed, as expected from the inequalities (\ref{eqn:rho_lower_bound}), that the bounds are tight as $x \to 0$ and $x \to \infty$, and that acceptance rates get lower as $|\lambda|$ increases. In all cases though they indicate that only large jumps (large $x$ values) will be rejected, and that at some point all jumps below a certain $x$ value are highly likely to be accepted on average.

\subsection{Simulation of GIG processes with $|\lambda|<0.5$}

The simple bound of Theorem \ref{Lemma_1} cannot be straightforwardly applied to simulation the GIG process for  $|\lambda| <0.5$. Instead we use the more sophisticated piecewise bounds of Theorem \ref{Lemma_2} and Corollary \ref{cor_piecewise}, which give us the dominating point process (\ref{GIG_bound_lam_le_0_5}) that can be split into two independent point processes $N_1$ and $N_2$ as follows:
\[
N_1:\,\,\,\,\,\frac{ e^{-x\gamma^2/2}}{\pi^2x^{1+|\lambda|}}\frac{(2\delta^2)^{|\lambda|}\gamma(|\lambda|,z_0^2x/(2\delta^2))\frac{\Gamma(|\lambda|)\sqrt{\text{Ga}}
(z||\lambda|,x/(2\delta^2))}{\gamma(|\lambda|,z_0^2x/(2\delta^2))}}{H_0z_0^{2|\lambda|-1}}{\cal I}_{z<z_0}
\]
\[
N_2:\,\,\,\,\,\frac{ e^{-x\gamma^2/2}}{\pi^2x^{3/2}}\frac{(2\delta^2)^{0.5}\Gamma(0.5,z_0^2x/(2\delta^2))\frac{\Gamma(0.5)\sqrt{\text{Ga}}
(z|0.5,x/(2\delta^2))}{\Gamma(0.5,z_0^2x/(2\delta^2))}}{H_0}{\cal I}_{z\geq z_0}
\]

\noindent Each may be simulated using a thinned tempered stable process for $x$ and a truncated $\sqrt{Ga}$ density for $z$. Having simulated each pair $(x,z)$ they are accepted with probability equal to the ratio $Q_{GIG}(x,z)/Q^0_{GIG(x,z)}$. Owing to the piecewise form of the acceptance probability the two processes may be treated independently, including accept reject steps and the point process union taken at the final step to achieve the final GIG samples.

The sampling recipe for point process $N_1$ is:

\begin{enumerate}
    \item $N_1={\emptyset}$
    \item Use the thinned series method to simulate points $x_i$, $i=1,....$ from a point process with intensity function:

\[
    \frac{e^{-x\gamma^2/2}}{\pi^2x^{1+|\lambda|}}\frac{(2\delta^2)^{|\lambda|}\gamma(|\lambda|,z_0^2x/(2\delta^2))}{H_0z_0^{2|\lambda|-1}}
\]

\noindent which is a modified tempered $|\lambda|$-stable process, see Section \ref{sec:marginal_x} for details.
    \item For each $x_i$, simulate a $z_i$ from a truncated square-root gamma density:

\[
    \frac{\Gamma(|\lambda|)\sqrt{\text{Ga}}
(z||\lambda|,x_i/(2\delta^2))}{\gamma(|\lambda|,z_0^2x_i/(2\delta^2))}{\cal I}_{0<z<z_0}
\]

\item  With probability
    \[
    \frac{H_0}{|H_{|\lambda|}(z_i)|^2\left(\frac{z_i^{2|\lambda|}}{z_0^{2|\lambda|-1}}
\right)}
\]

\noindent accept $x_i$, i.e. set $N_1=N_1\cup x_i$, otherwise discard $x_i$.
\end{enumerate}

Similarly, for $N_2$, the sampling recipe for an independent point process is as follows:

\begin{enumerate}
    \item $N_2={\emptyset}$
    \item Use the thinned series method to simulate points $x_i$, $i=1,....$ from a point process with intensity function:
\[
    \frac{e^{-x\gamma^2/2}}{\pi^2x^{3/2}}\frac{(2\delta^2)^{0.5}\Gamma(0.5,z_0^2x/(2\delta^2))}{H_0}
\]
    \item For each $x_i$, simulate a $z_i$ from a truncated square-root gamma density:

\[
    \frac{\Gamma(0.5)\sqrt{\text{Ga}} (z|0.5,x_i/(2\delta^2))}{\Gamma(0.5,z_0^2x_i/(2\delta^2))}{\cal I}_{z\geq z_0}
\]
\item  With probability:

\[
    \frac{H_0}{z_i|H_{|\lambda|}(z_i)|^2}
\]

\noindent accept  $x_i$, i.e. set $N_2=N_2\cup x_i$, otherwise reject $x_i$
\end{enumerate}

Finally, the set $N=N_1\cup N_2$ is the series of points from the GIG point process (still neglecting the additional gamma process that is required for $\lambda>0$).

\subsubsection{Acceptance Rates}

 Theorem \ref{Lemma_2} is used once again to find lower bounds on the expected acceptance rates. In this case the bound to apply is $z|H_{\lambda}(z)|^2 \leq A(z)$.

For $N_1$ the average acceptance rate $\rho_1(x)$ is
\begin{align*}
    \rho_1(x)=E \left[ \frac{H_0}{|H_{|\lambda|}(z)|^2 \left( \frac{z^{2 |\lambda| }}{z_0^{2|\lambda|-1}}\right)} \right] = \int_0^{z_0}\frac{\Gamma(|\lambda|)\sqrt{\text{Ga}}
(z||\lambda|,x/(2\delta^2))}{\gamma(|\lambda|,z_0^2x/(2\delta^2))} \frac{H_0}{z|H_{|\lambda|}(z)|^2} \left(\frac{z_0}{z}\right)^{2|\lambda|-1}
dz
\end{align*}
which may be lower bounded by substituting the bound $z|H_{\lambda}(z)|^2 \leq A(z)$ from Theorem \ref{Lemma_2} and by direct integration to give:
\[
    \rho_1(x) \geq \frac{H_0 z_0^{2|\lambda|-1} \pi^2}{2^{2|\lambda|} \Gamma^2(|\lambda|)}
\]

Similarly, for $N_2$ the average acceptance rate $\rho_2(x)$ is

\begin{align*}
 \rho_2(x)=   E \left[ \frac{H_0}{z|H_{|\lambda|}(z)|^2} \right] = \int_{z_0}^{\infty} \frac{\Gamma(0.5)\sqrt{\text{Ga}} (z|0.5,x/(2\delta^2))}{\Gamma(0.5,z_0^2x/(2\delta^2))} \frac{H_0}{z|H_{|\lambda|}(z)|^2} dz
\end{align*}
and again lower bounds are obtained by substituting $z|H_{\lambda}(z)|^2 \leq A(z)$ from Theorem \ref{Lemma_2} and direct integration to give:
\[
\rho_2(x) \geq \frac{\pi H_0}{2}
\]

We note that both of these bounds are in fact independent of the value of $x$. If we use $B(z)$ to obtain an upper bound on the acceptance rates $\rho_1$ and $\rho_2$, both bounds are found to be $1$, i.e. no useful information is gleaned from the upper bound.

\subsection{Sampling from the marginal point process envelope}
\label{sec:marginal_x}

In steps 2 above we require simulation of the marginal point process for $x$ in the dominating bivariate point process. For $N_1$ this has intensity:

\begin{equation}
    Q_1(x) = \frac{e^{-x\gamma^2/2}}{\pi^2x^{1+|\lambda|}}\frac{(2\delta^2)^{|\lambda|}\gamma(|\lambda|,z_0^2x/(2\delta^2))}{H_0z_0^{2|\lambda|-1}} = \frac{e^{-x\gamma^2/2}}{\pi^2x}\frac{\gamma(|\lambda|,z_0^2x/(2\delta^2))}{(z_0^2 x/(2\delta^2))^{|\lambda|} H_0 z_0^{-1}}
\end{equation}

\noindent and we propose two possible methods for simulating this point process.

The first and most basic method uses the fact that the lower incomplete gamma function is upper bounded by the complete gamma function, i.e. $\gamma(|\lambda|,z_0^2x/(2\delta^2))\leq \Gamma(|\lambda|)$, which allows generation from $Q_1$ by thinning of a tempered stable process. We thus have the following upper bounding envelope as a tempered stable density:

\[
    Q_1(x)\leq \bar{Q}_1^1(x) = \frac{e^{-x \gamma^2/2} \Gamma(|\lambda|)}{\pi^2 x^{1+|\lambda|}} \frac{(2\delta^2)^{|\lambda|}}{H_0 z_0^{2|\lambda|-1}}
\]

\noindent This process may be routinely sampled by thinning of a positive tempered $|\lambda|$-stable process as described in \ref{TS_sample}.

Having generated points from the tempered stable envelope function the  process is thinned with probability:

\[
    \frac{Q_1(x)}{\bar{Q}_1^1(x)}=\frac{\gamma(|\lambda|,z_0^2x/(2\delta^2))}{\Gamma(|\lambda|)}
\]

\noindent Experimentally we found that the acceptance rates were quite low for this bound, meaning that fairly long tempered stable series had to be generated (but see note below about the case $\gamma=0$), so a more sophisticated bound was sought, as follows.

The second and more effective method employs the following bound on the lower incomplete gamma function (\cite{Neuman2013}, Theorem 4.1):
\[
    \frac{a\gamma(a,x)}{x^a} \leq \frac{(1+ae^{-x})}{(a+1)}
\]
and so
\begin{equation}
    \frac{\gamma(|\lambda|,z_0^2 x/(2\delta^2))}{(z_0^2 x/(2\delta^2))^{|\lambda|}} \leq \frac{(1+|\lambda|e^{-z_0^2 x/(2\delta^2)})}{|\lambda|(|\lambda|+1)}
\end{equation}

\noindent so that the point process intensity function is upper bounded by

\[
Q_1(x)\leq \bar{Q}^2_1(x)=\frac{e^{-x\gamma^2/2}}{\pi^2x}\frac{z_0(1+|\lambda|e^{-z_0^2 x/(2\delta^2)})}{|\lambda|(1+|\lambda|)H_0}
\]

\noindent Note that this is the intensity function of the union of two gamma processes, which may routinely be simulated using the methods of \cite{Rosinski_2001}. Specifically, the first gamma process has a shape parameter $a_1 = \frac{z_0}{\pi^2 |\lambda|(1+|\lambda|)H_0}$ and a rate parameter $\beta_1 = \frac{\gamma^2}{2}$. The second gamma process has a shape parameter $a_2 = \frac{z_0}{\pi^2 (1+|\lambda|)H_0}$ and a rate parameter $\beta_2 = \frac{z_0^2}{(2 \delta^2)} + \frac{\gamma^2}{2}$. We can simulate these two gamma processes independently and their union will be thinned according to the following probability:

\[
     \frac{Q_1(x)}{\bar{Q}_1^2(x)}=\frac{|\lambda| (1+|\lambda|)\gamma(|\lambda|, z_0^2 x/(2\delta^2))}{\left(\frac{z_0^2 x}{2 \delta^2} \right)^{|\lambda|}(1+|\lambda|e^{-z_0^2 x/(2\delta^2)}) }\geq \frac{ (1+|\lambda|)e^{-|\lambda|\frac{z_0^2x}{2\delta^2(1+|\lambda|)}}}{(1+|\lambda|e^{-z_0^2 x/(2\delta^2)}) }
\]
Note that this method does not work for $\gamma=0$ since the underlying point process cannot be simulated in this case; instead the first method with bound $\bar{Q}_1^1(x)$ must be used for the $\gamma=0$ case.

The second method for simulating $N_1$, using bounding function $\bar{Q}_1^2(x)$, is regarded as the superior approach for all parameter settings except for $\gamma=0$ since it has increased acceptance probabilities and also relies on an underpinning Gamma process rather than a $|\lambda|$-stable process as its basis, and the series representation of the Gamma process is known to converge very rapidly to zero compared to the $|\lambda|$-stable process \cite{Rosinski_2001,B_N_Shephard_2001}.

Similarly, for $N_2$ we  follow a thinning approach. The L\'{e}vy density for $N_2$ is:

\begin{equation}
    Q_2(x) = \frac{e^{-x\gamma^2/2}}{\pi^2x^{3/2}}\frac{(2\delta^2)^{0.5}\Gamma(0.5,z_0^2x/(2\delta^2))}{H_0}
\end{equation}

\noindent Using the complete gamma function as part of an upper bound for this density, we can sample $N_2$ as a tempered stable intensity $\frac{e^{-x\gamma^2/2} (2\delta^2)^{0.5} \Gamma(0.5)}{\pi^2H_0 x^{3/2}}$, and apply thinning with probability $\Gamma(0.5, z_0^2 x/(2\delta^2))/\Gamma(0.5)$. This is found to work well, but the rejection rates can be improved by more tightly upper bounding the term $\Gamma(0.5, z_0^2 x/(2\delta^2))$. The following simple bound is suitable, and valid for $0<s<1$:
\[
    \Gamma(s,x) \leq x^{s-1} e^{-x}
\]
 Using this bound we can write
\begin{equation}
    \Gamma(0.5, z_0^2x/(2\delta^2)) \leq \left( z_0^2x/(2\delta^2)\right)^{-1/2} e^{- z_0^2x/(2\delta^2)}
\end{equation}
so that the point process $N_2$'s intensity function is upper bounded by:
\[
    Q_2(x) \leq \bar{Q}_2(x) =  \frac{(2\delta^2) e^{- \frac{x}{2} \left( \gamma^2 + z_0^2/\delta^2 \right)}}{\pi^2x^{2}z_0H_0}
\]

\noindent which is a tempered stable process and the thinning probability is:
\[
    \frac{Q_2(x)}{\bar{Q}_2(x)} = \frac{z_0x^{0.5} \Gamma(0.5,z_0^2x/(2\delta^2))}{(2\delta^2)^{0.5} e^{-z_0^2 x/(2\delta^2)}}
\]
\section{Simulations}
In this section example simulations from the new method are generated and compared with a random variable generator for the GIG distribution \cite{Devroye_2014,Statovic_2017}. In the shot noise case 1000 terms are generated for each point process series representation and $10^6$ random realisations  are generated to produce Figs. \ref{hist_qq_m0_1}-\ref{hist_qq_0_3},  in which examples of the principal parameter ranges and edge case $\gamma=0$ are presented. In Figs. \ref{GIG_path_m0_1}-\ref{GIG_path_m0_4_gamma_0} example pathwise simulations are plotted for several parameter settings, drawing 30 independent realisations of the GIG process in each case. For cases where $\lambda>0$ the additional term in the L\'{e}vy density, $\frac{e^{-x\gamma^2/2}}{x}$, is generated as an independent Gamma process and the union of the these points with those from $Q_{GIG}$ is taken to generate the final simulated process.

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{hist_qq_m0_1.png}
\caption{Simulation comparison between the shot noise generated GIG process and GIG random variates, $\lambda=-0.1$, $\gamma=0.1$, $\delta=2$, $10^6$ random samples.}
 \label{hist_qq_m0_1}
\centering
\includegraphics[width=0.8\textwidth]{hist_qq_m0_4.png}
\caption{Simulation comparison between the shot noise generated GIG process and GIG random variates, $\lambda=-0.4$, $\gamma=0.5$, $\delta=1$, $10^6$ random samples.}
 \label{hist_qq_m0_4}
\end{figure}

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{hist_qq_m1.png}
\caption{Simulation comparison between the shot noise generated GIG process and GIG random variates, $\lambda=-1$, $\gamma=0.5$, $\delta=4$, $10^6$ random samples.}
 \label{hist_qq_m1}
 \includegraphics[width=0.8\textwidth]{hist_qq_m0_3_gamma_0.png}
\caption{Simulation comparison between the shot noise generated GIG process and GIG random variates, $\lambda=-0.3$, $\gamma=0$, $\delta=4$, $10^6$ random samples.}
 \label{hist_qq_m0_3_gamma_0}
\end{figure}

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{hist_qq_m1_gamma_0.png}
\caption{Simulation comparison between the shot noise generated GIG process and GIG random variates, $\lambda=-1$, $\gamma=0$, $\delta=4$, $10^6$ random samples.}
 \label{hist_qq_m1_gamma_0}
\end{figure}

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{hist_qq_1.png}
\caption{Simulation comparison between the shot noise generated GIG process and GIG random variates, $\lambda=1$, $\gamma=0.4$, $\delta=4$, $10^6$ random samples.}
 \label{hist_qq_1}
 \includegraphics[width=0.8\textwidth]{hist_qq_0_3.png}
\caption{Simulation comparison between the shot noise generated GIG process and GIG random variates, $\lambda=0.3$, $\gamma=0.5$, $\delta=2$, $10^6$ random samples.}
 \label{hist_qq_0_3}
\end{figure}

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{GIG_path_m0_1.png}
\caption{Pathwise simulations of the GIG process, $\lambda=-0.1$, $\gamma=0.5$, $\delta=3$.}
 \label{GIG_path_m0_1}
 \includegraphics[width=0.8\textwidth]{GIG_path_m_1.png}
\caption{Pathwise simulations of the GIG process, $\lambda=-1$, $\gamma=0.5$, $\delta=3$ (log-scale).}
 \label{GIG_path_m_1}
\end{figure}

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{GIG_path_1.png}
\caption{Pathwise simulations of the GIG process, $\lambda=1$, $\gamma=0.4$, $\delta=4$.}
 \label{GIG_path_1}
 \includegraphics[width=0.8\textwidth]{GIG_path_m0_4_gamma_0.png}
\caption{Pathwise simulations of the GIG process, $\lambda=-0.4$, $\gamma=0$, $\delta=1$ (log-scale).}
 \label{GIG_path_m0_4_gamma_0}
\end{figure}

\section{Discussion}
This paper has presented a generic simulation methodology for GIG processes. The methods are simple and efficient to implement and have good acceptance rates, which will make them of use in applications for practitioners. Moreover, we provide code in Matlab and Python in order to give researchers immediate access to the methods.

Simulation of GIG processes opens up the possibility of simulation and inference for many more complex processes; in particular, a direct extension takes the sampled GIG process and generates other processes within the generalised shot-noise methodology \cite{Rosinski_2001}:
\[
W(t)=\sum_{i=1}^{\infty} H(U_{i},\Gamma_{i}){\cal I}(V_{i}\leq t)
\]
where $U_i$ are iid uniform random variates and $H(u,\gamma)$ a non-increasing function of $\gamma$.
Of particular interest will be the mean-variance mixture of Gaussians, with:
\[
H(U,\gamma)\sim {\cal N}(\mu_{W} h(\gamma),\sigma_{W}^{2} h(\gamma))
\]
and $J_{i}=h(\Gamma_{i})$ is the $i$th ordered jump of the simulated GIG process.
This approach, which is an exactly equivalent process to the time-changed Brownian motion description of \cite{B_N_1997}, leads directly to a simulation method for the Generalised Hyperbolic (GH) process, and its conditionally Gaussian form will enable inference for these processes, using a conditionally Gaussian approach similar in spirit to \cite{Lemke_Godsill_2015, Godsill_Riabiz_Kont_2019}.
Our continued work on these processes will study these inferential approaches with the GIG/GH model and also their use as driving processes for stochastic differential equations, again extending the approach of \cite{Godsill_Riabiz_Kont_2019} to these more general classes of process.

\printbibliography

\end{document}


\section{Introduction}

There is an increasing research interest in developing the next generation video compression technologies. While the traditional video compression methods~\cite{wiegand2003overview, sullivan2012overview} have achieved promising performance by using the hand-designed modules (\textit{e.g.}, block-based motion estimation, Discrete Cosine Transform (DCT)) to reduce spatial and temporal redundancy, these modules cannot be optimized in an end-to-end fashion based on large-scale video data.

The recent deep video compression works~\cite{lu2019dvc,wu2018video,Agustsson2020space,abdelaziz2019neural,hu2020imporving,lu2020content,liu2020mlvc} have achieved impressive results by applying the deep neural networks within the hybrid video compression framework.
Currently, most works only rely on pixel-level operations~(\textit{e.g.,} motion estimation or motion compensation) for reducing redundancy.
For example, pixel-level optical flow estimation and motion compensation are used in \cite{lu2019dvc,lu2020content,hu2020imporving,lu2020anendtoend} to reduce temporal redundancy, and pixel-level residual is further compressed by using the auto-encoder style network.
However, this pixel-level paradigm suffers from the following three limitations.
First, it is difficult to produce accurate pixel-level optical flow information, especially for videos with complex non-rigid motion patterns.
Second, even we can extract sufficiently accurate motion information, the pixel-level motion compensation process may introduce additional artifacts.
Third, it is also a non-trivial task to compress pixel-level residual information.

Given robust representation ability of deep features for various applications, it is desirable to perform motion compensation or residual compression in the feature space and more effectively reduce spatial or temporal redundancy in videos.
Furthermore, the recent progress in deformable convolution~\cite{Dai2017DeformableCN,Wang2019EDVRVR} has shown it is feasible to align two consecutive frames in the feature space.
Based on the learned offset maps of convolution kernels (\textit{i.e.}, the so-called dynamic kernels), deformable convolution can decide which positions are sampled from the input feature.
By using deformable convolution based on dynamic kernels, we can better cope with more complex non-rigid motion patterns between two consecutive frames, which can thus improve the motion compensation results and also alleviate the burden for the subsequent residual compression module.
Unfortunately, it is still a non-trivial task to seamlessly incorporate the feature space operations and deformable convolution into the existing hybrid deep video compression framework and train the whole network in an end-to-end fashion.

In our work, instead of following the existing works~\cite{lu2019dvc,hu2020imporving,agustsson2019generative,liu2020mlvc} to adopt the pixel-level operations as in the traditional video codecs, we propose a new feature-space video coding network (referred to as FVC) by reducing the spatial-temporal redundancy in the feature space.
Specifically, we first estimate motion information (\textit{i.e.}, the offset maps for convolution kernels in deformable convolution), based on the extracted representations from two consecutive frames.
Then the offset maps are compressed by using the auto-encoder style network and the reconstructed offset maps will be used in the subsequent deformable convolution operation to generate the predicted feature for more accurate motion compensation.
After that, we adopt another auto-encoder style network to compress the residual feature between the original input feature and the predicted feature.
Finally, a multi-frame feature fusion module is proposed to combine a set of reference features from multiple previous frames for better frame reconstruction.

When compared with the state-of-the-art learning based video compression methods~\cite{lu2020content,hu2020imporving}, we perform all operations in the feature space for more accurate motion estimation and motion compensation, in which we can seamlessly incorporate deformable convolution into the hybrid deep video compression framework.
As a result, we can alleviate the errors introduced by inaccurate pixel-level operations like motion estimation/compensation and achieve better video compression performance.
Our contributions are summarized as follows:

\begin{itemize}
  \item We propose a new learning based video compression framework, which performs all operations including motion estimation, motion compensation and residual compression in the feature space.

  \item For effective motion compensation in the feature space, we use deformable convolution to ``warp'' the reference feature from the previous reconstructed frame and more accurately predict the feature of the current frame, in which the corresponding offset maps are compressed by using the auto-encoder style network.

  \item We propose a new multi-frame feature fusion module based on the non-local attention mechanism, which combines the features from multiple previous frames for better reconstruction of the current frame.

  \item Our framework FVC achieves the state-of-the-art performance on four benchmark datasets including HEVC, UVG, VTL and MCL-JCV, which demonstrates the effectiveness of our proposed framework.

\end{itemize}

\section{Related Work}

\subsection{Image Compression}
In the past decades, the traditional image compression methods like JPEG~\cite{wallace1992jpeg}, JPEG2000~\cite{taubman2002jpeg2000} and BPG~\cite{bellard2015bpg} have been proposed to reduce spatial redundancy, in which the hand-crafted techniques, such as DCT, are exploited for achieving high compression performance. Recently, the learning based image compression methods have attracted increasing attention. The RNN-based image compression methods~\cite{toderici2015variable, toderici2017full, johnston2018improved} are firstly introduced to progressively compress images. Other methods~\cite{balle2016end, balle2018variational, minnen2018joint, theis2017lossy} adopted the auto-encoder structure to first encode images as latent representations, and then decode the latent representations from the feature space to the pixel space, which have achieved the state-of-the-art performance for image compression.

\subsection{Video Compression}
A number of video compression standards~\cite{wiegand2003overview, sullivan2012overview} have been proposed in the past few years. Most methods follow the hybrid coding structure, where the motion compensation and residual coding techniques are used to reduce the redundancy in both spatial and temporal domains. Recently, learning based video compression \cite{wu2018video,lu2019dvc,rippel2019learned,habibian2019video,abdelaziz2019neural,liu2020mlvc,lu2020content,lu2020anendtoend, hu2020imporving, feng2020learned,Agustsson2020space,yang2020learning,cheng2019learning,xu2019non,lombardo2019deep,klopp2020utilising,yang2020learningh} has become a new research direction.
Following the traditional hybrid video compression framework, Lu \etal~\cite{lu2019dvc} proposed the first end-to-end optimized video compression framework, in which all the key components in H.264/H.265 are replaced with deep neural networks.
Lin \etal~\cite{liu2020mlvc} used multiple frames in different modules to further remove the redundancy, while Hu \etal~\cite{hu2020imporving} proposed a resolution-adaptive optical flow compression method by automatically deciding the optimal resolution for each frame and each block.

Most of the existing approaches~\cite{lu2019dvc,liu2020mlvc,hu2020imporving} have to estimate the pixel-level optical flow maps and compress the corresponding pixel-level residual information. However, it may be difficult to produce reliable pixel-level flow maps or residual information, which often degrades the compression performance of learning based video codecs.
To address this problem, some recent works~\cite{abdelaziz2019neural,feng2020learned} proposed to calculate the feature-level residual maps for video compression.
In contrast to these works~\cite{abdelaziz2019neural,feng2020learned}, in our work, we propose to perform all operations in the feature space, which leads to better video compression performance.

\begin{figure*}[t]
\centering
\includegraphics[width=0.95\linewidth]{figures/overview.pdf}
    \caption{Overview of our proposed video compression framework. Given an input frame $X_t$, we first encode the input frame to produce the input feature $F_t$. The \textbf{Deformable Compensation} module consists of three steps (\textit{i.e.}, motion estimation, motion compression and motion compensation). Specifically, the offset map $O_t$ between the features $F_t$ and $F_{t-1}^{ref}$ from the previous reconstructed frame is estimated and then used as motion information. The offset map is then compressed and the reconstructed offset map $\hat{O}_t$ is employed for motion compensation by using the deformable convolution operation. The residual feature $R_t$ between the input feature $F_t$ and the predicted feature $\Bar{F}_t$ is compressed in the \textbf{Residual Compression} module. Additionally, the \textbf{Multi-frame Feature Fusion} module is proposed to fuse the initial reconstructed feature $\tilde{F}_t$ and the features $F_{t-1}^{ref},F_{t-2}^{ref}$ and $F_{t-3}^{ref}$ from multiple previous frames to produce the final reconstructed feature $\hat{F_t}$. Finally, the reconstructed frame $\hat{X}_t$ is generated after going through the frame reconstruction module.}
\label{fig:overview}
\end{figure*}

\subsection{Deformable Convolution}
Recently, Dai \etal~\cite{Dai2017DeformableCN} proposed to use deformable convolution together with the learnt offset maps to enhance the modeling capability of convolution neural networks, which has achieved promising results in several video-related tasks like action recognition~\cite{Li2020SpatiotemporalD3} and video super-resolution~\cite{Tian2020TDANTA, Wang2019EDVRVR}.
In contrast to these works, our work is the first to investigate how to employ deformable convolution in learning based video compression. Considering that there are multiple complex components in our hybrid video compression system, it is a non-trivial task to develop an end-to-end optimized video codec by seamlessly incorporating deformable convolution and simultaneously compressing the corresponding offset maps and other features.

\section{Methodology}

\subsection{Overview}

Let $X = \{X_1, X_2,...,X_{t-1}, X_t,...\}$ denote a video sequence, where $X_t$ is the original video frame at the current time step.
In our video compression system, the objective is to produce the high quality reconstructed frame $\hat{X}_t$ at any given bit-rate. As shown in Fig.~\ref{fig:overview}, all the modules in our proposed framework including deformable compensation, residual compression and multi-frame feature fusion are performed in the feature space.
The overview architecture of our proposed framework is summarized as follows,

\begin{figure}[t]
\centering
\includegraphics[width=0.9\linewidth]{figures/fcoder.pdf}
    \caption{The network structure of (a) our Feature Extraction module and (b) our Frame Reconstruction module with the details of each Resblock are shown in (c). ``Conv(N,K,S)" and ``Deconv(N,K,S)" represent the convolution and deconvolution operations with the output channel, the kernel size and the stride as $N$, $K\times K$ and $S$, respectively.}
\label{fig:edcoder}
\end{figure}

\textbf{Feature Extraction.}
To produce the representations in the feature space, the original input frame $X_t$ and the previous reconstructed frame $\hat{X}_{t-1}$ are encoded as the feature representations $F_t$ and $F^{ref}_{t-1}$, respectively.
As shown in Fig.~\ref{fig:edcoder}(a), the feature extraction module uses a convolution layer with the stride of 2, which is then followed by several residual blocks to produce the feature representation.

\textbf{Deformable Compensation.}
This procedure consists of three steps: motion estimation, motion compression and motion compensation.
Specifically, based on $F_t$ and $F^{ref}_{t-1}$, we perform motion estimation by using a lightweight network, and the output offset map $O_t$ will be compressed by using the newly proposed motion compression network before being transmitted to the decoder side. Finally, given the reconstructed offset map $\hat{O}_t$ and the feature $F_{t-1}^{ref}$, we can generate the predicted feature $\bar{F}_{t}$ by using deformable convolution in the motion compensation procedure. More details can be found in Section~\ref{subsec:compensation}.

\textbf{Residual Compression.}
The residual feature $R_t$ between the input feature $F_t$ and the predicted feature $\Bar{F}_t$ will be compressed by using an auto-encoder style network for better reconstruction. After adding the reconstructed residual feature $\hat{R}_t$ back to the predicted feature $\Bar{F}_t$, we produce the initial reconstructed feature representation $\Tilde{F}_t$.

\textbf{Multi-frame Feature Fusion.}
In this procedure, the extracted feature representations $F^{ref}_{t-1}$, $F^{ref}_{t-2}$ and $F^{ref}_{t-3}$ from three previous reconstructed frames as well as the initial reconstructed feature representation $\Tilde{F}_t$ are fused to produce the final reconstructed feature $\hat{F}_t$.
More details will be introduced in Section~\ref{subsec:fusion}.

\textbf{Frame Reconstruction.}
As shown in Fig.~\ref{fig:edcoder}(b), the feature decoder with several residual blocks and a deconvolution layer will transform the final reconstructed feature $\hat{F}_t$ to the reconstructed frame $\hat{X_t}$.

\textbf{Entropy Coding.}
The quantized features from the motion compression and residual compression modules will be transformed into the bit-streams by performing entropy coding. During the training process, we use a bit estimation network to estimate the number of bits. More details will be described in section~\ref{subsec:compress}.

\subsection{Deformable Compensation}
\label{subsec:compensation}

The previous video compression frameworks~\cite{lu2019dvc, abdelaziz2019neural, hu2020imporving} rely on the optical flow estimation network and the pixel-level motion compensation network, which may lead to inaccurate frame prediction results and thus bring extra redundancy to the subsequent residual compression module.
To this end, we perform motion compensation in the feature space and generate the predicted features by using the deformable convolution operation.

\begin{figure}[t]
\centering
\includegraphics[width=0.9\linewidth]{figures/compensation.pdf}
    \caption{The network structure of our \textbf{Deformable Compensation} module, which consists of (a) the motion estimation submodule, (b) the motion compression submodule and (c) the motion compensation submodule for reducing redundancy.}
\label{fig:compensation}
\end{figure}

Given the features $F^{ref}_{t-1}$ and $F_t$ respectively from the previous reconstructed frame and the current frame, the deformable compensation procedure aims at generating the predicted feature $\bar{F}_t$ at the current time step. The whole network architecture is shown in Fig.~\ref{fig:compensation}.
Specifically, we first produce the deformable offset map $O_t$ by feeding  $F^{ref}_{t-1}$ and $F_t$ into a 2-layer motion estimation network.
Inspired by~\cite{feng2020learned}, we propose an auto-encoder style network to compress this offset map $O_t$, where both the encoder and the decoder consist of a set of Resblocks~\cite{He2016DeepRL}. The offset map $O_t$ is transformed to the latent space through the encoder and then quantized. After that, the decoder will convert the quantized latent representations back to the reconstructed offset map $\hat{O}_t$.

Finally, in the motion compensation procedure, the deformable convolution layer takes $F^{ref}_{t-1}$ as the input and then performs the deformable convolution operation by using the corresponding filters with the aid of $\hat{O}_t$.
In this way, we can better compress videos with complex non-rigid motion patterns by using the learned dynamic offset kernels and produce a more accurate warped feature.
To generate a more accurate predicted feature, we further refine the output feature from the deformable convolution layer by using two convolution layers and eventually produce the final predicted feature $\bar{F}_t$.

\begin{figure}[t]
\centering
\includegraphics[width=0.7\linewidth]{figures/deformable.pdf}
    \caption{Illustration of Deformable Convolution. For the output channel of the convolution layer, ``G", ``2", ``9" denote the channel group ($G=8$), two directions (\textit{i.e.}, the horizontal and vertical directions) of the offset map and the size of each kernel ($3\times3$), respectively.
\label{fig:deformable}
\end{figure}

\textbf{Deformable Convolution.}
The network structure of our deformable convolution layer~\cite{Dai2017DeformableCN} is shown in Fig.~\ref{fig:deformable}. Given the reference feature map and the corresponding offset map $O$, we aim to generate the predicted feature.
For each kernel, the corresponding offsets are used to control the sampling location in the reference feature map, and then the feature values from the corresponding locations in the reference feature map are fused to generate one feature value in the predicted feature map.
When compared with the pixel-level warping operation based on motion compensation as used in the previous video compression approach~\cite{lu2019dvc}, our deformable compensation module provides more flexibility by using different sampling locations, which can better cope with complex non-rigid motion patterns and improve the compensation performance.
In our implementation, we divide the channels of the reference feature into G groups (G=8 in this work) and use a shared offset map for each group of channels to better learn the offset maps and improve the deformable compensation results.

\subsection{Multi-frame Feature Fusion}
\label{subsec:fusion}

\begin{figure}[t]
\centering

\subfigure[The overall network structure for multi-frame feature fusion.]{\includegraphics[width=\linewidth]{figures/fusion.pdf}\label{subfig:non-fusion}}
\subfigure[Non-local attention mechanism.]{\includegraphics[width=0.5\linewidth]{figures/attention.pdf}\label{subfig:nonatt}}

\centering
    \caption{Illustration of our multi-frame feature fusion module.
\label{fig:fusion}
\end{figure}

After the deformable compensation and residual compression procedures, we produce the initial reconstructed feature $\tilde{F}_t$ by combining the predicted feature $\bar{F}_t$ and the reconstructed residual feature $\hat{R}_t$.
As shown in Fig.~\ref{fig:overview}, to generate more accurate final reconstructed feature $\hat{F}_t$,  a multi-frame feature fusion module is proposed to exploit temporal context information from the feature representations $F^{ref}_{t-1}$, $F^{ref}_{t-2}$, $F^{ref}_{t-3}$ of multiple previous frames.

The detailed network architecture is shown in Fig.~\ref{subfig:non-fusion}.
Specifically, we first produce the predicted feature representations $\bar{F}^{ref}_{t-1}$, $\bar{F}^{ref}_{t-2}$ and $\bar{F}^{ref}_{t-3}$ by using the shared feature-space deformable compensation module.
The non-local attention mechanism is then used to refine the predicted feature representation based on the similarity between the initial reconstructed feature $\tilde{F}_t$ and the predicted representations $\bar{F}^{ref}_{t-1}$, $\bar{F}^{ref}_{t-2}$ and $\bar{F}^{ref}_{t-3}$.
It is worth mentioning that we also refine $\tilde{F}_t$ itself by using the non-local attention mechanism based on the so-called self-attention mechanism.
Finally, we concatenate all these refined feature representations and use another convolution layer to generate the final reconstructed feature $\hat{F}_t$.

The network architecture of the deformable compensation module is the same as that in section~\ref{subsec:compensation}.
As for the non-local attention block, we provide the detailed architecture in Fig.~\ref{subfig:nonatt}.  Here we take $\bar{F}^{ref}_{t-1}$ as an example to describe how to produce the refined feature $\hat{F}_{t-1}^{ref}$ after performing the attention operation guided by $\tilde{F}_t$, in which we assume the dimension of these two features $\bar{F}^{ref}_{t-1}$ and $\tilde{F}_t$ is $w \times h \times c$. For each spatial feature vector $f_{i,j}$ from $\tilde{F}_t$ at the location $(i,j)$, whose size is $1 \times 1 \times c$, we find a collocated patch $f^{ref}_{i,j}$ with the size of $p \times p \times c$ from $\bar{F}^{ref}_{t-1}$. Then we calculate the similarity between $f_{i,j}$ and $f^{ref}_{i,j}$ by performing the convolution operation along the channel dimension and then produce the corresponding attention map after performing the softmax operation, which is further used to re-weight all spatial positions in $f^{ref}_{i,j}$. After that, we generate the refined feature vector $\hat{f}^{ref}_{i,j}$ at the location $(i,j)$. We repeat this procedure for each spatial location in $\tilde{F}_t$ and then produce the refined feature $\hat{F}^{ref}_{t-1}$. Finally, we concatenate all refined features $\hat{F}_{t-3}^{ref}$, $\hat{F}_{t-2}^{ref}$, $\hat{F}_{t-1}^{ref}$ and $\hat{F}_{t}^{ref}$ and perform the convolution operation to fuse these refined features and produce the final reconstructed feature $\hat{F}_t$ by adding the initial reconstructed feature $\tilde{F}_t$ back.

\subsection{Residual Compression and Other Details}
\label{subsec:compress}

As shown in Fig.~\ref{fig:overview}, in addition to offset map compression, we also need to compress the residual feature map. To simplify the whole system, we adopt the same network architecture as that used for the offset feature map compression (see Fig.~\ref{fig:compensation}(b)).

For the whole learning based video compression framework, we use the bit-rate estimation network to generate the bit-rate in the training stage. In our implementation, we employ the hyperprior entropy model in \cite{minnen2018joint} for accurate bit-rate estimation. To reduce the computational complexity, the time-consuming auto-regressive model is not used in our approach.

To optimize the whole model in an end-to-end manner, it is also required to design a differentiable quantization operation. In our approach, we follow the method in \cite{balle2018variational} and approximate the quantization operation by adding the uniform noise in the training stage.  In the inference stage, we directly use the \textit{rounding} operation.

\subsection{Loss Function}
\label{subsec:training}
In our proposed framework, we optimize the following Rate Distortion~(RD) trade-off:
\begin{equation}
\vspace{-1mm}
    RD = R + \lambda D =  R_o + R_r + \lambda d(X_t, \hat{X}_t),
\vspace{-1mm}
\end{equation}
where $R_o$ and $R_r$ denote the numbers of bits used to encode the offset map $O_t$ and the residual feature map $R_t$, respectively. $d(X_t, \hat{X}_t)$ denotes the distortion between the input frame $X_t$ and the reconstructed frame $\hat{X}_t$, where $d(\cdot)$ represents the mean square error or MS-SSIM~\cite{wang2003multiscale}. $\lambda$ is a hyper parameter used to control the rate-distortion trade-off.

\vspace{-1mm}

\section{Experiments}
\vspace{-1mm}

\begin{figure*}[t]
  \centering
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/VTL_psnr.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/VTL_msssim.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/HEVCClass_B_psnr.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/HEVCClass_B_msssim.pdf}
  \end{minipage}
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/UVG_psnr.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/UVG_msssim.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/HEVCClass_C_psnr.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/HEVCClass_C_msssim.pdf}
  \end{minipage}
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/MCL_psnr.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/MCL_msssim.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/HEVCClass_D_psnr.pdf}
  \end{minipage}%
  \begin{minipage}[c]{0.25\textwidth}
  \centering
    \includegraphics[width=\textwidth]{figures/performance/HEVCClass_D_msssim.pdf}
  \end{minipage}
    \caption{The experimental results on the VTL, UVG, MCL-JCV and HEVC Class B, Class C, Class D datasets. When using MS-SSIM for performance evaluation, we additionally perform the fine-tune operation in our FVC$^\#$ by using MS-SSIM as the distortion loss.}
  \label{fig:result}
\end{figure*}

\subsection{Experimental Setup}
\textbf{Training Dataset.}
We use the Vimeo-90K dataset~\cite{xue2019video} in the training stage, which contains 89,800 video clips with each video having 7 frames with the resolution of $448 \times 256$. We randomly crop the video sequences into the resolution of $256\times256$ before the training process.

\textbf{Testing Datasets.}
To evaluate the performance of our FVC, we use the video sequences in four datasets including the HEVC~\cite{sullivan2012overview}, UVG~\cite{UVGdataset}, MCL-JCV~\cite{wang2016mcl} and VTL~\cite{VTLdataset}. The HEVC datasets contain 16 videos in Class B, C, D and E with different resolutions from $416\times240$ to $1920\times1080$. The UVG dataset contains seven high frame rate videos with the resolution of $1920\times1080$, in which the difference between the neighboring frames is small. The MCL-JCV dataset is widely used for video quality evaluation, which consists of thirty 1080p video sequences. For the VTL dataset, we use the first 300 frames from the high-resolution video sequences with the resolution of $352\times 288$ for performance evaluation.

\textbf{Evaluation Metrics.}
We use bpp (bit per pixel) to measure the average number of bits used for motion coding and residual coding for one pixel in each frame. We use PSNR and MS-SSIM\cite{wang2003multiscale} to evaluate the distortion between the reconstructed frame and the ground-truth frame and the PSNR/MS-SSIM of each video sequence is produced by calculating the average PSNR/MS-SSIM over all reconstructed frames.

\textbf{Implementation Details.}
We train four models with different $\lambda$ values ($\lambda$=256, 512, 1024, 2048).  A two-stage training scheme is used to train our model. In the first training stage, we train the model without using the multi-frame feature fusion module for 2,000,000 steps at the learning rate of 5e-5. In the second stage, the multi-frame feature fusion module is included and we first train the overall framework with the learning rate of 5e-5 for 400,000 steps, and then optimize the model with the reduced learning rate of 5e-6 for 100,000 steps. When using MS-SSIM for performance evaluation, we further fine-tune the model from Stage 2 for 80,000 steps by using MS-SSIM as the distortion loss to achieve better MS-SSIM results. Our method is implemented based on PyTorch with CUDA support. All the experiments are conducted on the machine with a single NVIDIA 2080TI GPU (11GB memory). We set the batch size as 4 and use the Adam optimizer~\cite{kingma2014adam}. It takes about 4 days, 3 days and 12 hours for the first training stage, the second training stage and the fine-tuning stage, respectively.

\subsection{Experimental Results}

\begin{table}[!t]
\centering
\small

            \caption{BDBR(\%) results of DVC~\cite{lu2019dvc}, EA~\cite{Agustsson2020space}, LU~\cite{lu2020content}, HU~\cite{hu2020imporving} and our proposed method FVC when compared with H.265~\cite{sullivan2012overview} on different datasets. Negative values in BDBR indicate bit-rate savings while positive values indicate more bit-rate costs.}\setlength{\tabcolsep}{2mm}{
    \begin{tabular}{|l|c|c|c|c|c|c|c|}
    \hline
        & DVC & EA & LU & HU & FVC\\
    \hline
        HEVC Class B & 2.97   &   -    & -15.92 & -14.91          & \textbf{-23.75}\\
    \hline
        HEVC Class C & 20.65  &   -    & -3.78  & 1.76            & \textbf{-14.18}\\
    \hline
        HEVC Class D & 14.08  &   -    & -8.29  & -1.77           & \textbf{-18.39}\\
    \hline
        UVG          & 8.45   &  -9.75 & -7.34  & -13.27          &\textbf{-28.71}\\
    \hline
        VTL          & -10.92 &   -    & -16.85 & -20.17          &\textbf{-28.10}\\
    \hline
        MCL-JCV      & 13.94  &  -1.52 &  4.75  & -13.71          &\textbf{-22.48}\\
    \hline
    \end{tabular}}
    \label{tab:BDBR}
\end{table}

\textbf{Settings of the Baseline Methods.}
In this section, we provide the experimental results to demonstrate the effectiveness of our proposed method FVC on four benchmark datasets HEVC~\cite{sullivan2012overview}, UVG~\cite{UVGdataset}, VTL~\cite{VTLdataset} and MCL-JCV~\cite{wang2016mcl}. We compare our method with other state-of-the-art methods including the traditional methods~\cite{wiegand2003overview,sullivan2012overview} and recently proposed learning based methods (DVC~\cite{lu2019dvc}, AD\_ICCV19~\cite{abdelaziz2019neural}, EA\_CVPR20~\cite{Agustsson2020space}, LU\_ECCV20~\cite{lu2020content} and HU\_ECCV20~\cite{hu2020imporving}). For the traditional method H.265~\cite{sullivan2012overview}, we follow the command line in \cite{hu2020imporving} and use FFmpeg with the \textit{default} mode.
For fair comparison, we additionally report the results of ``DVC*", which is an enhanced version of DVC by using the same motion and residual encoder/decoder modules as those in our deformable compensation module (see Fig.~\ref{fig:compensation}).
Following the previous methods~\cite{lu2019dvc,hu2020imporving,lu2020content}, we set the GoP size as 10 for the HEVC datasets and 12 for other datasets. We use the same I-frame compression method as in H.265 to reconstruct I-frame. For our ``FVC$^\#$", we further perform the fine-tune operation according to the MS-SSIM based rate-distortion trade-off.

\textbf{Results.}
In Table~\ref{tab:BDBR}, we report the BDBR~\cite{bjontegaard2001calculation} results of different video compression methods when compared with H.265 on the HEVC, UVG, VTL and MCL-JCV datasets.
Specifically, our approach saves more than 18\% bit rate in terms of the overall results on all benchmark datasets. It is obvious that our approach outperforms DVC and other recent state-of-the-art learning based video compression approaches.
For example, when compared with H.265, our proposed FVC saves 18.39\% bit-rate on the HEVC Class D dataset while the corresponding bit-rate savings are 8.29\% and 1.77\% for the recent approaches LU\_ECCV20~\cite{lu2020content} and HU\_ECCV20~\cite{hu2020imporving}.

We provide the RD curves of different compression methods in Fig.~\ref{fig:result}, it is noted that our method outperforms the baseline method DVC~\cite{lu2019dvc} and the enhanced version DVC* by a large margin on all datasets. We would like to highlight that DVC* directly compresses the pixel-level optical flow maps and the pixel-level residual maps by using the same compression network as our proposed method FVC, so the performance improvement clearly demonstrates the effectiveness of our proposed method. When compared with the traditional method H.265, our method achieves better results in terms of PSNRs at all bit-rates.
Our method also outperforms all other baseline methods in terms of MS-SSIM.

\subsection{Ablation Study and Analysis}

\begin{figure}[t]
\centering
    \begin{minipage}[t]{0.5\linewidth}
    \centering
    \subfigure[]{\includegraphics[height=1.23in]{figures/ablation/HEVC_Class_D_mff_psnr.pdf}\label{fig:abmff}}

    \end{minipage}%
    \begin{minipage}[t]{0.5\linewidth}
    \centering
    \subfigure[]{\includegraphics[height=1.23in]{figures/ablation/HEVC_Class_D_fea_psnr.pdf}\label{fig:abfea}}

    \end{minipage}
  \caption{Ablation study on the HEVC Class D dataset. (1) \textbf{FVC}: Our proposed method FVC. (2) \textbf{FVC w/o NLA}: FVC without adopting the non-local attention (NLA) mechanism in the multi-frame feature fusion (MFF) module. (3) \textbf{FVC w/o MFF (also referred to as FVC-basic)}: FVC without using the whole MFF module. (4) \textbf{FVC-basic (FS-motion \& PS-residual)} and (5) \textbf{FVC-basic (PS-motion \& PS-residual)} are two variants of FVC-basic by only performing residual compression at the pixel space and by performing both motion-related operations and residual compression at the pixel space, respectively. Note the motion-related operations and residual compression in FVC-basic are both performed in the feature space, which are referred to as FS-motion and FS-residual, respectively. We also refer to the corresponding pixel-space operations in DVC* as PS-motion and PS-residual.}

\label{fig:ablation}
\end{figure}

\textbf{Effectiveness of Different Components.} As shown in Fig.~\ref{fig:ablation},  we take the HEVC Class D dataset as an example to demonstrate the effectiveness of different modules in our proposed framework FVC.
As shown in Fig.~\ref{fig:abmff}, to demonstrate the effectiveness of the non-local attention (NLA) module, we remove NLA in the multi-frame feature fusion (MFF) module and the results of FVC w/o NLA drop nearly 0.2dB when compared with our complete model (\textit{i.e.}, FVC).
We also introduce the basic model of our approach~(\textit{i.e.}, FVC w/o MFF or FVC-basic), where the multi-frame feature fusion module is removed in our complete model. When compared with our complete model (\textit{i.e.}, FVC), the result of our basic model FVC-basic after removing the MFF module drops 0.5dB at 0.3bpp~(see the green and black curves). These experimental results demonstrate the effectiveness of our newly proposed MFF module equipped with the NLA mechanism for fusing the features from multiple previous frames.

To verify the effectiveness of our proposed feature-level operations, in Fig.~\ref{fig:abfea} we report the results of two variants of our FVC-basic.
\textbf{(1) FVC-basic (FS-motion \& PS-residual)}. The predicted feature $\bar{F}_t$ after our deformable compensation module is converted as the predicted frame by using our frame reconstruction module, which is then used as the input to the pixel-space residual compression module in DVC*.
\textbf{(2) FVC-basic~(PS-motion \& PS-residual}. We perform both motion-related operations and residual compression at the pixel space, which is conceptually the same as DVC* except some subtle differences in implementation details.

It is noted that FVC-basic outperforms FVC-basic~(FS-motion \& PS-residual) by 0.3dB at 0.38bpp, which indicates it is beneficial to perform residual compression at the feature space. Furthermore, FVC-basic~(FS-motion \& PS-residual) achieves 1.2dB improvement at 0.4bpp when compared with FVC-basic~(PS-motion \& PS-residual), which demonstrates it is also necessary to perform motion compensation at the feature space.

\begin{figure}[t]
\centering
  \includegraphics[width=0.57\linewidth]{figures/ablation/motionanalysis.pdf}
  \caption{Average PSNR(dB) over all predicted frames and bpps used to encode motion information of our FVC and DVC* on the HEVC Class C dataset.}
\label{fig:motionab}
\end{figure}

\textbf{Analysis of Deformable Compensation.}
For better comparison of the motion compensation module in the feature space and the pixel space, we also take the predicted feature $\bar{F}_t$ as the input of our frame reconstruction module to produce the predicted frame and evaluate the PSNR of the predicted frame and the corresponding bpp for compressing motion information. As shown in Fig.~\ref{fig:motionab}, the predicted frames of our proposed FVC achieves 1.75dB improvement at 0.017bpp on the HEVC Class C dataset when compared with the predicted frames of DVC*. It should be mentioned that DVC* uses the same compression method based on the same auto-encoder style network as our proposed method FVC, so the result clearly demonstrates the effectiveness of our proposed deformable compensation module.

\begin{figure}[!t]
\centering
\begin{minipage}[t]{0.5\linewidth}
\centering
\subfigure[Frame No.6 (Ground-truth)]{\includegraphics[width=0.9\linewidth]{figures/offsetflow/C0033input.png}\label{fig:frame6}}
\end{minipage}%
\begin{minipage}[t]{0.5\linewidth}
\centering
\subfigure[Optical flow map (DVC*)]{\includegraphics[width=0.9\linewidth]{figures/offsetflow/flow.png}\label{fig:opticalflow}}
\end{minipage}
\begin{minipage}[t]{0.5\linewidth}
\centering
\subfigure[Offset map 1 of DC (Ours)]{\includegraphics[width=0.9\linewidth]{figures/offsetflow/offset1.png}\label{fig:offset1}}
\end{minipage}%
\begin{minipage}[t]{0.5\linewidth}
\centering
\subfigure[Offset map 2 of DC (Ours)]{\includegraphics[width=0.9\linewidth]{figures/offsetflow/offset2.png}\label{fig:offset2}}
\end{minipage}
\begin{minipage}[t]{0.3333333333333\linewidth}
\centering
\subfigure[One patch (GT)]{\includegraphics[width=0.99\linewidth]{figures/offsetflow/gtflag.png}\label{fig:gtball}}
\end{minipage}%
\begin{minipage}[t]{0.3333333333333\linewidth}
\centering
\subfigure[Flow (26.68/0.0346)]{\includegraphics[width=0.99\linewidth]{figures/offsetflow/flowflag.png}\label{fig:flowball}}
\end{minipage}%
\begin{minipage}[t]{0.333333333333\linewidth}
\centering
\subfigure[DC (26.95/0.0223)]{\includegraphics[width=0.99\linewidth]{figures/offsetflow/dcflag.png}\label{fig:dcball}}
\end{minipage}
\caption{Visualization of the results by using optical flow based motion compensation (b,f) and our deformable compensation (c,d,g) for the 6th frame (a) in the \textit{RaceHorses} sequence from the HEVC Class C dataset. (b): the reconstructed optical flow map from the motion compression network of DVC*. (c,d): two representative reconstructed offset maps used for deformable convolution (DC) in our deformable compensation module. Visualization results of one zoomed-in patch from: the ground-truth (e), the flow based motion compensation module from DVC* (f) and our deformable compensation module (g), in which both PSNR and bpps for coding motion information are reported.}
\label{fig:offsetvis}
\end{figure}

\textbf{Visualization of Deformable Compensation Results.}
In Fig.~\ref{fig:offsetvis}, we take the 6th frame in the \textit{RaceHorses} sequence from the HEVC Class C dataset as an example and visualize motion information and the motion compensation results. We visualize the optical flow map used for motion compensation in DVC* (see Fig.~\ref{fig:opticalflow}) and two representative offset maps (from the total number of $72$ offset maps) used for deformable convolution in our work (see Fig.~\ref{fig:offset1} and Fig.~\ref{fig:offset2}). It can be observed that the learned offset maps in our work encode similar motion patterns as the optical flow map from DVC*.  We also observe that the motion compensation result for one patch after using our proposed deformable compensation (DC) module are visually more similar to the ground-truth patch when compared with that by using the optical flow based motion compensation method in DVC*. In practice, we can achieve 0.27dB gain by only using 65\% bpp for motion coding (see Fig.~\ref{fig:gtball}, Fig.~\ref{fig:flowball} and Fig.~\ref{fig:dcball}).

\textbf{Running Time and Model Complexity.}
The total number of parameters in our proposed framework is about 26M, in which the parameters from the compression network used for offset maps and residual feature maps take more than 24M.
We use the videos with the resolution of $1920\times1080$ to evaluate the inference time on the machine with a single 2080TI GPU (11GB Memory). The coding time of our proposed frameworks FVC and FVC-basic as well as DVC and DVC* are 548ms, 201ms, 460ms and 709ms, and the corresponding BDBRs on the HEVC Class D dataset by using H.265 as the anchor method are -18.39\%, -7.09\%, 14.08\% and 4.31\%. From the results, we observe that our FVC-basic outperforms DVC and DVC* in terms of both efficiency and effectiveness. In addition, our proposed framework FVC achieves the best BDBR result with 347ms used for the multi-frame feature fusion module, which can be accelerated by using a simpler fusion module in our future work.

\vspace{-1mm}
\section{Conclusion}
\vspace{-1mm}
In this work, we have proposed a new framework FVC for deep video compression in the feature space, which consists of the deformable compensation module, the feature-level residual compression module and the multi-frame feature fusion module. The deformable compensation module first predicts the offset maps as motion information, which are then compressed by using the newly proposed motion compression module and the reconstructed offset maps will be finally used in deformable convolution for motion compensation. The proposed multi-frame feature fusion module takes multiple feature representations from the current frame and the previous frames and uses the deformable compensation and the non-local attention mechanism to refine the initial reconstructed feature for better frame reconstruction. By performing all the operations in the feature space, our framework achieves promising results on the HEVC, UVG, VTL and MCL-JCV datasets

\noindent\textbf{Acknowledgement}
This work was supported by the National Key Research and Development Project of China (No. 2018AAA0101900).

{\small
\bibliographystyle{ieee_fullname}