\section{Introduction}

In this paper we study the ergodic theory of systems with random switchings. Such a system
can be described in terms of a finite family of vector fields. We assume that at any given time the
evolution is driven by one of these vector fields, and at random times the driving vector field changes to another
one from the same family. Systems of this nature arise naturally in applications and we refer to the recent
monograph~\cite{Yin:MR2559912} for motivation and extensive bibliography.

 Many long-term asymptotic properties of dynamical systems or random dynamical systems can be
described in terms of invariant distributions.
The existence of invariant measures often can be derived using the Lyapunov function technique that helps to
establish recurrence properties or tightness, see, e.g.,\cite[Sections~3.3--3.4]{Yin:MR2559912}.

The uniqueness and absolute
continuity of invariant distributions
are often related to each other and more subtle, especially in the case that we consider in this paper where no
diffusion is involved, and the only source of randomness is the random sequence of driving vector fields. Although
some claims have been scattered through the literature, no general result is known, see, e.g.,~\cite[Section
8.5.2]{Yin:MR2559912}.

The goal of this paper is to close this gap and obtain new general conditions that guarantee uniqueness and absolute
continuity of invariant measures for systems with random switchings. The two conditions that we suggest are
formulated in terms of Lie algebras associated to the driving vector fields. They are close analogues of the
classical H\"ormander condition guaranteeing absolute continuity of transition densities of hypoelliptic diffusions.
In the diffusion context, this result is usually derived from the variational analysis of diffusion paths known as
Malliavin calculus, see, e.g.,~\cite[Chapter VIII]{Bass:MR1483890},\cite{Bell:MR2250060},\cite{Nualart:MR2200233}.

In fact, the central part of this paper is the analysis of transition probabilities of switched systems. Under the
first of our conditions, we prove that all transition probabilities for the system have nontrivial absolutely
continuous components. The second condition is more general, and it allows to prove the existence of absolutely
continuous components not for the transition probabilities themselves, but for their time averages. The extraction of
these absolutely continuous components is largely based on classical control theory results that can be found  in
Chapter~$3$ of~\cite{Jurdjevic:MR1425878}. These control theory results rely on earlier work by
Chow~\cite{Chow:MR0001880}, Sussmann and Jurdjevic~\cite{Sussmann-Jurdjevic:MR0338882}, and
Krener~\cite{Krener:MR0383206}.
Our conditions and the
structure of our proofs match those of~\cite{Jurdjevic:MR1425878}, where the nondegeneracy of certain
maps is exploited to establish the accessibility property. We use the same nondegeneracy to prove absolute continuity,
and one can
interpret our result as filling the control theory with probabilistic content. In fact, the idea to use the
geometric control theory approach to establish regularity of Markov transition kernels along with ergodic
properties is not new, see, e.g., \cite{Agrachev-Kuksin-Sarychev-Shirikyan:MR2329509} where controllability
of the 2D Navier--Stokes system was used to prove the absolute continuity of finite-dimensional projections of
transition kernels.

The article is organized as follows: In Section~\ref{sec:definitions} we introduce the setting, necessary notation and
notions from differential geometry and geometric control theory. We also state the main result on uniqueness and
absolute continuity of invariant measures, and two central auxiliary results on regularity of transition probabilities
each based on one of the H\"ormander type assumptions. We prove these regularity results in
Sections~\ref{sec:AC-component-1-proof} and~\ref{sec:AC-component-2-proof}. In
Section~\ref{sec:AC-of-ergodic-measures} we prove that any ergodic measure has to be absolutely continuous if
its support contains a point where hypoellipticity holds. Section~\ref{sec:uniqueness-proof} contains the
proof of the main result: if the hypoellipticity holds at a point that can be approached from any initial point using
the given vector fields as admissible controls, then there exists at most one invariant distribution, and this
distribution has to be absolutely continuous. In Section~\ref{sec:examples}, we apply the main result to a switching system on the $n$-dimenisonal torus and a switching system involving two Lorenz vector fields.

{\bf Acknowledgments:} The idea to study invariant densities for systems with switchings emerged after a discussion
of ``blinking systems'' with Leonid Bunimovich and Igor Belykh, and we would like to thank them. We are also thankful to
L.Bunimovich for his comments on the Lorenz system.
We are grateful
to Martin Hairer and especially Jonathan Mattingly for stimulating discussions of other possible approaches to the main
results of this paper. The partial support from NSF through a CAREER Award DMS-0742424 is gratefully acknowledged by YB.
\section{Definitions, Notation, and Main Results} \label{sec:definitions}

\subsection{The dynamics} \label{sec:dynamics}

We consider a finite collection $D$ of smooth and forward complete vector fields
on an $n$-dimensional $C^\infty$-manifold $M$.

We denote these vector fields by $u_i,i\in S=
\{1,\ldots,k\}$.
Each vector field $u$ in $D$ induces an ordinary differential equation of the
form
\begin{equation*}
\dot{x}(t) = u(x(t)).
\end{equation*}
This differential equation is uniquely solvable if equipped with
an initial condition
\begin{equation*}
x(0) = \xi\in M,
\end{equation*}
 and forward completeness means that the solution trajectories are well-defined
for all times $t>0$.

We can define a stochastic process
$X=(X_t)_{t \geq 0}$ on $M$ in the following way: Given an initial state $i\in S$ and an initial
value $\xi \in M$, $X_t$ follows the trajectory generated by the vector field $u_i$  and initial
condition $\xi$ for an
exponentially distributed random time with parameter $\lambda_i>0$. Then a new state is selected at random from
$S\setminus\{i\}$, and,
for another exponentially distributed random time, $X_t$ follows the new vector field corresponding to that state.
Iterating this construction  we obtain a piecewise smooth
trajectory $(X_t)_{t\ge 0}$ defined for all positive times and driven by one of the vector fields from $D$
between any two switchings.
We assume that (i) all the inter-switching times are exponentially distributed and
independent conditioned on the sequence of driving vector
fields, (ii) the parameter $\lambda_j$ of the exponential time between any two switches depends only on the current
state $j$,
 and (iii) the probabilities of
switchings between any two states are positive.

It is convenient to keep track of the driving vector fields at all times. We
define $A_t\in S$ as the index of the driving vector field at time $t$, also
referred to as the regime or state at time $t$. It is a Markov process with continuous time and finitely
many states. Its trajectories are right-continuous and piecewise constant.

Although $X$ alone is not a Markov process, the joint process $(X,A)$ is Markov. We denote elements of the
associated Markov family, i.e., the distribution on paths emitted at $(\xi,i)\in M \times S$  and generated by
the iterative random procedure above, by $\mathsf{P}_{\xi,i}$, and the corresponding transition probability measures by
$\mathsf{P}_{\xi,i}^{t}$,  $t \geq 0$. The transition probability measures are
defined on the product $\sigma$-algebra $\mathcal{B}(M) \otimes \mathcal{P}(S)$, where $\mathcal{B}(M)$ is the
Borel $\sigma$-algebra on $M$ and $\mathcal{P}(S)$ is the power set of $S$. We write $\mathsf{E}_{\xi,i}$ for expectation with
respect to $\mathsf{P}_{\xi,i}$.

Let us recall that if the initial distribution of the Markov process $(X,A)$ is $\mu$, then  the distribution of the
process at time $t$ is given by the measure $\mu \mathsf{P}^t$ on $M\times S$ defined by
\begin{equation}
 \mu \mathsf{P}^t(E\times\{j\}) = \sum_{i=1}^{k} \int_{M} \mathsf{P}_{\xi,i}^t(E\times\{j\})\, \mu(d\xi\times\{i\}).
\label{eq:convolution}
\end{equation}

A probability measure $\mu$ on $M \times S$ is called
invariant for $(\mathsf{P}^t)$ if
$\mu=\mu \mathsf{P}^t$ for all $t \geq 0$.

The main goal of this paper is to give conditions on $D$ that would guarantee absolute
continuity and uniqueness of an invariant measure of the Markov semigroup $(\mathsf{P}^t)=(\mathsf{P}^t)_{t\ge 0}$. The fairly general
conditions that we suggest are formulated in geometric terms, and we proceed to introduce the
necessary
definitions and notation.

\subsection{Auxiliary definitions and notation}\label{sec:diff_geometry}

Let $V(M)$ denote the set of real smooth vector fields on the manifold $M$, and let $C^{\infty}(M)$ denote
the set of real-valued smooth functions on $M$. As explained above, we assume that $D$ is
contained in $V(M)$. Any element of $V(M)$ corresponds uniquely to a derivation on $C^{\infty}(M)$, that is to
a linear operator $\delta$ on $C^{\infty}(M)$ satisfying the Leibniz rule
\begin{equation*}
\delta(f \cdot g) = \delta(f) \cdot g + f \cdot \delta(g).
\end{equation*}
The Lie bracket of two vector fields $u$ and $v$ in $V(M)$ is defined as the vector field
\begin{equation*}
[u,v](f):=u(v(f)) - v(u(f))
\end{equation*}
for test functions $f$ in $C^{\infty}(M)$. The set $V(M)$ equipped with the bilinear operator $[.,.]$ becomes
a Lie algebra over the reals. A subset of $V(M)$ is called involutive if it is closed under taking the Lie bracket. An
involutive subspace of $V(M)$ is called a subalgebra of~$V(M)$.

The smallest subalgebra of $V(M)$ that contains $D$ is denoted $\mathcal{I}(D)$. The
derived algebra $\mathcal{I}'(D)$ is the smallest algebra containing
Lie brackets of vector fields in $\mathcal{I}(D)$. We have $\mathcal{I}'(D)\subset \mathcal{I}(D)$, but
$\mathcal{I}'(D)$ might not contain any elements of $D$ and may therefore be strictly contained in $\mathcal{I}(D)$.
Further, we define $\mathcal{I}_{0}(D)$ as the set of vector fields of the form
\begin{equation*}
v + \sum_{i=1}^{k}{\lambda_{i}u_{i}},
\end{equation*}
where $v \in \mathcal{I}'(D)$, $u_1,\ldots,u_k \in D$ and $\sum_{i=1}^{k}{\lambda_{i}}=0$. Finally, we set
\begin{equation*}
\mathcal{I}(D)(\xi):=\{u(\xi):\ u \in \mathcal{I}(D)\}
\end{equation*}
and
\begin{equation*}
\mathcal{I}_{0}(D)(\xi):=\{u(\xi):\ u \in \mathcal{I}_{0}(D)\}
\end{equation*}
for any $\xi \in M$. The sets $\mathcal{I}(D)(\xi)$ and $\mathcal{I}_{0}(D)(\xi)$ are finite-dimensional
vector spaces.

\medskip

Our main results will be based on the following assumptions that can naturally be called hypoellipticity conditions
in analogy with H\"ormander's theory.
We say that a point $\xi\in M$ satisfies Condition~A if $\dim \mathcal{I}_{0}(D)(\xi)=n$.
We say that a point $\xi\in M$
satisfies Condition~B if  $\dim \mathcal{I}(D)(\xi)=n$.

The set of points satisfying Condition~A is open and so is the set of
points satisfying Condition~B.
\medskip

For our absolute continuity results we will need a reference measure on $M$ that will play the role of Lebesgue
measure. As a smooth manifold, $M$ can be endowed with a Riemannian metric.
The metric tensor can be used to define measures on coordinate patches of $M$.
One can use then a partition of
unity in a standard way (see, e.g., \cite[Section 7]{Taylor:MR2245472})
to construct a Borel measure on $M$ whose
pushforward to $\mathbb{R}^{n}$ under any chart map is equivalent to Lebesgue measure. We call the measure on $M$
obtained through this construction Lebesgue measure, denote it by $\lambda^{M}$, and use
it as the main reference measure, often omitting ``with respect to Lebesgue measure'' when writing about absolute
continuity.
The product of the Lebesgue measure on $M$ and counting measure on $S$ will be called
the Lebesgue measure on
$M\times S$. We denote the Lebesgue
measure on
$\mathbb{R}^{m}$ by~$\lambda^{m}$.

\medskip

It remains to introduce the flows generated by vector fields in $D$ and the concept of reachability.

For $i\in S$, we denote the flow function of the vector field $u_i$ by $\Phi_i$. Due to forward completeness of $u_i$, the
flow function is uniquely defined for all $t>0$ and $\eta\in M$ by
\begin{align*}
 \frac{d}{dt}\Phi_i(t,\eta)&=u_i(\Phi_i(t,\eta)),\\
 \Phi_i(0,\eta)&=\eta.
\end{align*}

For $m\in\mathbb{N}$, we will consider vectors
${\bf t}=(t_1,\ldots,t_m)$ of waiting times between subsequent switches and vectors ${\bf i}=(i_1,\ldots,i_m)$ of
driving states during these waiting intervals. We will restrict ourselves to positive waiting times, but it can also be useful
(see \cite{Sussmann-Jurdjevic:MR0338882} and \cite{Jurdjevic:MR1425878}) to admit flows backwards in
time.

We write $\mathbb{R}_{+}$ to denote the positive real line $(0;\infty)$.

For  ${\bf t}=(t_1,\ldots,t_m)\in \mathbb{R}_{+}^{m}$ and ${\bf i}=(i_1,\ldots,i_m)\in S^{m}$, we define
\begin{equation*}
\Phi_{{\bf i}}({\bf t},\xi):=\Phi_{i_m}(t_m,\Phi_{i_{m-1}}(t_{m-1},\ldots\Phi_{i_1}(t_1,\xi))\ldots)
\end{equation*}
as the cumulative flow along the trajectories of $u_{i_1},\ldots,u_{i_m}$ with starting point $\xi\in M$.

The transition probabilities $\mathsf{P}_{\xi,i}^{t}$ can be expressed in terms of cumulative flows. We do not specify
these straightforward relations in order to avoid heavy notation.

A point $\eta\in M$ is called $D$-reachable from a point $\xi\in M$ if there exist a time vector ${\bf
t}$ with
positive components and a vector~${\bf i}$ of driving states such that
\begin{equation*}
\eta = \Phi_{{\bf i}}({\bf t},\xi).
\end{equation*}
If the components of ${\bf t}$ sum up to $t$, we say that $\eta$ is $D$-reachable from $\xi$ at time~$t$.

For $\xi \in M$ and $t>0$, let $L_{t}(\xi)$ denote the set of $D$-reachable points from $\xi$ at time~$t$, and
let $L(\xi)=\bigcup_{t>0} L_{t}(\xi)$ denote the set of $D$-reachable points from $\xi$.  The points in
the closure $\overline{L(\xi)}$ can be called $D$-approachable from $\xi$.
Let $L=\bigcap_{\xi \in M} \overline{L(\xi)}$ denote the set of points that are $D$-approachable from all
other
points.

\subsection{Main results} \label{sec:absolute-continuity-invariant}

The following is the main theorem of this paper.

\begin{theorem} \label{thm:uniqueness}
Suppose Hypoellipticity Condition~B is satisfied at some $\xi \in L$.
If $(\mathsf{P}^t)$ has an invariant measure, then it is unique and absolutely continuous with respect
to the Lebesgue measure on $M\times S$.
\end{theorem}

\begin{remark}\rm
Of course, Theorem~\ref{thm:uniqueness} remains true if we replace $L$ by any of its subsets. For example, if
one of the vector fields in $D$ has a minimal global
attractor, then it is sufficient to check hypoellipticity for some point of the attractor.
\end{remark}

Uniqueness of invariant distributions is tightly connected to the regularity of the Markov semigroup. Various
aspects of regularity in connection with ergodicity have been studied in the literature: the existence of minorizing
kernels, the strong Feller property, etc.
The main task in the proof of Theorem~\ref{thm:uniqueness} is to establish regularity for
transition probabilities under Hypoellipticity Condition~B. However, we begin with a much stronger regularity property that
can be established under the stronger Hypoellipticity Condition~A.

\begin{theorem}\label{thm:AC-component-1}
If Condition~A is satisfied at a point $\xi\in M$, then for any $i \in
S$ and any $t > 0$, the transition kernel $\mathsf{P}_{\xi,i}^{t}$ has a nonzero absolutely
continuous component with respect to Lebesgue measure on $M \times S$.
\end{theorem}

Under the weaker Condition~B it may happen that none of the transition
probability measures $\mathsf{P}_{\xi,i}^t$, $t > 0$, has a nonzero absolutely continuous component. For example, let $M$~be
the
$n$-dimensional torus $\mathbb{T}^n=\mathbb{R}^n/\mathbb{Z}^n$, and let
$D=\{u_1,\ldots,u_n\}$ be the standard basis in $\mathbb{R}^{n}$. Fix an arbitrary time $t > 0$. The set of
points $D$-reachable from the origin at time $t$ is the image of
\begin{equation*}
\biggl\{(s_1,\ldots,s_n) \in [0;\infty)^{n}: \quad \sum_{j=1}^{n} s_j =t  \biggr\}
\end{equation*}
under the covering map $\mathbb{R}^n\to\mathbb{T}^n$,
and has Lebesgue measure zero, so $\mathsf{P}_{\xi,i}^t$ is a purely singular measure.

Nevertheless, Condition~B guarantees that time averages of transition probabilities have
nontrivial absolutely continuous components.  Specifically, we will establish this for
the resolvent probability kernel $\mathsf{Q}_{\xi,i}$ defined by
\begin{equation}
\label{eq:resolvent}
\mathsf{Q}_{\xi,i}(E\times\{j\}):= \int_{\mathbb{R}_{+}} e^{- t}\,\mathsf{P}_{\xi,i}^t(E\times\{j\}) dt.
\end{equation}
The resolvent kernels are useful in the study of invariant distributions due to the following straightforward result.
\begin{lemma}\label{lem:Q-invariance}
If a measure $\mu$ is $(\mathsf{P}^t)$-invariant it is also $(\mathsf{Q})$-invariant, i.e.,
$\mu=\mu \mathsf{Q}$, where the convolution $\mu\mathsf{Q}$ is defined analogously to~\eqref{eq:convolution}.
\end{lemma}

\begin{theorem}\label{thm:AC-component-2}
If Condition B is satisfied at some point $\xi\in M$, then for any $i\in S$,
the measure $\mathsf{Q}_{\xi,i}$ defined by~\eqref{eq:resolvent}
has a nonzero absolutely continuous component with respect to Lebesgue measure on $M \times S$.
\end{theorem}

\bigskip

Although the convergence of transition probabilities to the invariant measures is out of
the scope of the present paper and will be addressed elsewhere, our analysis suggests that under the assumptions of
Theorem~\ref{thm:uniqueness} and Condition~A at $\xi$ this convergence holds true while Condition~B at $\xi$ implies
only Ces\`aro convergence.

At the heart of our proofs of Theorems~\ref{thm:AC-component-1} and~\ref{thm:AC-component-2} are classical results
from  geometric control theory that can be found in~\cite{Jurdjevic:MR1425878}. The statements we present are derived
from Theorems~$3.1$,~$3.2$, and~$3.3$ in~\cite{Jurdjevic:MR1425878}. Analogous results for the special case of analytic
vector fields on a real analytic manifold are first stated in~\cite[Theorems~$3.1$ and
$3.2$]{Sussmann-Jurdjevic:MR0338882}. In their paper, Sussmann and Jurdjevic were able to build on prior
work~\cite{Chow:MR0001880} by Chow who considered symmetric families of analytic vector fields.
Krener generalized these results to $C^{\infty}$-vector fields in~\cite{Krener:MR0383206}.

Recall that a regular point of a function $f:\mathbb{R}^{m} \to M$ is a point ${\bf t} \in \mathbb{R}^{m}$ such that the differential $Df({\bf t})$ has full rank.
If $Df({\bf t})$ has deficient rank, ${\bf t}$ is called a critical point of~$f$.

\begin{theorem} \label{thm:Jurdjevic-1} Assume that Condition~A holds at
some $\xi\in M$.
Then:
\begin{enumerate}
 \item For any $i,j\in S$,
there are an integer $m>n$
and  a vector ${\bf i} \in S^{m+1}$ with $i_1=i$ and $i_{m+1}=j$ such that for any $t>0$ the mapping $f_\mathbf{i}:\mathbb{R}_{+}^{m} \to M$
defined by
\begin{equation}
\label{eq:flow_for_transition_probability}
f_\mathbf{i}(t_1,\ldots,t_{m})=\Phi_{{\bf i}}\biggl(t_1,\ldots,t_{m},t-\sum_{l=1}^{m} t_l, \xi \biggr)
\end{equation}
has a nonempty open set of regular points in the simplex
\begin{equation*}
\Delta_{t,m}:=\biggl\{(t_1,\ldots,t_m) \in \mathbb{R}_{+}^{m}:\ \sum_{l=1}^{m} t_l < t \biggr\}.
\end{equation*}
\item The interior of $L(\xi)$ is nonempty and dense in $L(\xi)$.
\end{enumerate}
\end{theorem}

\begin{theorem} \label{thm:Jurdjevic-2}
Assume that Condition~B holds at some $\xi\in M$.
Then:
\begin{enumerate}
 \item For any $i,j\in S$,
there are an integer $m>n$
and  a vector ${\bf i} \in S^{m+1}$ with $i_1=i$ and $i_{m+1}=j$ such that for any $t>0$ the mapping $F_\mathbf{i}:\mathbb{R}_{+}^{m+1} \to M$
defined by
\begin{equation*}
F_\mathbf{i}(t_1,\ldots,t_{m+1})=\Phi_{{\bf i}}(t_1,\ldots,t_{m+1},\xi)
\end{equation*}
has a nonempty open set of regular points in $\Delta_{t,m+1}$.
\item The interior of $L(\xi)$ is nonempty and dense in $L(\xi)$.
\end{enumerate}
\end{theorem}

Condition~A is stronger than Condition~B, so it is not surprising that the conclusion of
Theorem~\ref{thm:Jurdjevic-1} implies the conclusion of Theorem~\ref{thm:Jurdjevic-2}.

The first statement of Theorem~\ref{thm:Jurdjevic-2} corresponds to Theorem~$3.1$ in~\cite{Jurdjevic:MR1425878}, which reads as follows: Under the assumptions of Theorem~\ref{thm:Jurdjevic-2}, any neighborhood $U$ of $\xi$ contains points that are normally accessible from $\xi$ at arbitrarily small times. A point $\eta$ in $M$ is called normally accessible from $\xi$ at time $t>0$ if there exist vectors $\mathbf{i} \in S^{m+1}$ and $(\hat{t}_1,\ldots,\hat{t}_{m+1}) \in \Delta_{t,m+1}$ such that $F_\mathbf{i}(\hat{t}_1,\ldots,\hat{t}_{m+1}) = \eta$ and the differential $DF_\mathbf{i}(\hat{t}_1,\ldots,\hat{t}_{m+1})$ has full rank. It's worth pointing out, though, that in~\cite{Jurdjevic:MR1425878} only one sequence~$\mathbf{i}$ resulting in $F$ with a regular point is constructed. But since the flow generated by any vector field is a family of diffeomorphisms, and since the set of points satisying Condition~B is open, one can append any indices in front or at the back of that sequence without destroying the desired properties, and thus recover this part of Theorem~\ref{thm:Jurdjevic-2} as we state it.

The fact that the interior of $L(\xi)$ is nonempty and dense in $L(\xi)$ follows from Theorem~$3.2.a$ in~\cite{Jurdjevic:MR1425878}.

Theorem~\ref{thm:Jurdjevic-1} essentially follows from applying Theorem~$3.1$ (\cite{Jurdjevic:MR1425878}) to $\mathbb{R} \times M$ and
vector fields ${\bf 1} \oplus u_i, i\in S$, where
\begin{equation*}
({\bf 1} \oplus u)(r,\xi):= (1,u(\xi)),\quad (r,\xi)\in \mathbb{R} \times M,
\end{equation*}
and ${\bf 1}$ is the unit vector field on $\mathbb{R}$ corresponding to
the derivation $\partial/\partial r$ and identically equal to~$1$ in the natural coordinates on $\mathbb{R}$.

\section{Proof of Theorem~\ref{thm:AC-component-1}} \label{sec:AC-component-1-proof}

We need to prove that for any $t>0$ and $i \in S$, the measure $\mathsf{P}^t_{\xi,i}$ is not singular.

\medskip

For any finite sequence ${\mathbf{i}}$ of indices in $S$ with initial index $i$ (we will call these sequences
admissible), let $C_{{\mathbf{i}}}$ be the event that the driving vector fields up to time $t$ appear in the order determined by ${\mathbf{i}}$. Since
$\mathsf{P}_{\xi,i}(C_{\mathbf{i}})>0$ for any admissible $\mathbf{i}$
it suffices to find an admissible sequence $\mathbf{i}$ such that
$\mathsf{P}_{\xi,i}^t(\cdot| C_{\mathbf{i}})$ is not singular.
We claim that this holds true for the the sequence $\mathbf{i}$ provided by
Theorem~\ref{thm:Jurdjevic-1}. According to
Theorem~\ref{thm:Jurdjevic-1}, there is an admissible sequence $\mathbf{i}=(i_1,i_2,\ldots,i_{m+1})$ with $i_1 = i$
such that the function
$f_\mathbf{i}$
has a regular point in $\Delta_{t,m}$. Since the set of regular points of a differentiable function is open in its domain, the function $f_\mathbf{i}$ is regular in a nonempty open set $B \subset \Delta_{t,m}$.

Let $T_1,T_2,\ldots,T_{m+1}$ be independent and exponentially distributed random variables such that $T_j$ has
parameter
$\lambda_{i_j}$ for $1\leq j \leq m+1$.

On $C_\mathbf{i}$ we have $A_t=i_{m+1}$, and the distribution of $X_t$ under $\mathsf{P}_{\xi,i}(\cdot | C_{\mathbf{i}})$ coincides
with the distribution of $f_\mathbf{i}(T_1,\ldots,T_m)$ conditioned on the event
\begin{equation}
\label{eq:event_R}
R=\biggl\{\sum_{j=1}^{m} T_j < t \le \sum_{j=1}^{m+1} T_j \biggr\}.
\end{equation}
The distribution of the random vector $(T_1,\ldots,T_m)$ conditioned on $R$,
is equivalent to the uniform distribution on the simplex
\begin{equation*}
\Delta_{t,m} := \biggl\{(t_1,\ldots,t_m) \in \mathbb{R}_{+}^{m}: \quad \sum_{j=1}^{m}t_j < t \biggr\}.
\end{equation*}
Now the theorem directly follows from the following result:
\begin{lemma}\label{lem:pushforward_full_rank} Let $n,m\in\mathbb{N}$, $n\le m$.
Suppose that $B$ and $\Delta$ are nonempty open sets in~$\mathbb{R}^m$, $B$ is dense in $\Delta$,
and $M$ is an $n$-dimensional
smooth manifold.
If $f:\Delta\to M$ is
differentiable on $\Delta$ and all points in $B$ are regular for $f$, then for any
absolutely continuous
probability measure $\mu$ on $\Delta$ satisfying $\mu(B)>0$, its
pushforward
$\mu f^{-1}$ is not singular with respect to~$\lambda^M$.
\end{lemma}

We will prove this lemma only for $M=\mathbb{R}^n$.  Modifying the proof for the general case using
coordinate patches on $M$ amounts only to notational differences.

We will use the following statement (see, e.g., Proposition~4.4
in~\cite{Davydov-et-al:MR1604537}):
\begin{lemma}\label{lem:pushforward_for_nonzero_det}
 Let  $f:B \to\mathbb{R}^m$ be a Borel function a.e.-differentiable on an open set $B \subset\mathbb{R}^m$
and satisfying  $\lambda^m\{\mathbf{t}\in B:\, \det Df(\mathbf{t})=0\}=0$. If $\mu\ll \lambda^m$, then
$\mu f^{-1}\ll\lambda^m$, and
\[
 \frac{d(\mu f^{-1})}{d\lambda^m}(\mathbf{s})=\sum_{\mathbf{t}\in B: f(\mathbf{t})=\mathbf{s}} |\det Df(\mathbf{t})|^{-1}\frac{d\mu}{d\lambda^m}(\mathbf{t})
\]

\end{lemma}

\bpf[Proof of Lemma~\ref{lem:pushforward_full_rank}]We must prove that for any $E \subset \mathbb{R}^n$ with $\lambda^n(E)=0$,
\begin{equation}
\label{eq:not_singular}
\mu\{\mathbf{t}\in\Delta:\, f(\mathbf{t})\in E\}<1.
\end{equation}
For any $\mathbf{t} \in B$, we can find some $n$
columns of $Df(\mathbf{t})$ (without loss of generality, first~$n$ columns) that are linearly
independent.
For $\rho:B \to \mathbb{R}^n \times \mathbb{R}^{m-n}$ defined by
\[
\rho:\mathbf{t}=(t_1,\ldots,t_m) \mapsto (f(\mathbf{t}), t_{n+1},\ldots,t_m),
\]
and any $\mathbf{t}\in B$, we have
\begin{equation*}
\det D\rho(\mathbf{t} ) \neq 0.
\end{equation*}
 Since $E \times \mathbb{R}^{m-n}$ is a set of measure zero, Lemma~\ref{lem:pushforward_for_nonzero_det} implies
that
\begin{equation*}
0 = \lambda^{m}\{\mathbf{t} \in B:\, \rho(\mathbf{t}) \in E \times \mathbb{R}^{m-n}\} = \lambda^{m}\{\mathbf{t} \in B:\, f(\mathbf{t}) \in E\}.
\end{equation*}
Therefore, $\mu\{\mathbf{t} \in B:\, f(\mathbf{t}) \in E\}=0$, and
\[
 \mu\{\mathbf{t}\in\Delta:\, f(\mathbf{t})\in E\}= \mu\{\mathbf{t}\in \Delta \setminus B:\, f(\mathbf{t})\in E\}\le 1 -\mu(B)<1.
\]
so \eqref{eq:not_singular} is established and the proof is complete.
{{\hfill $\Box$ \smallskip}}

\section{Proof of Theorem~\ref{thm:AC-component-2}}  \label{sec:AC-component-2-proof}

 We need to show that $\mathsf{Q}_{\xi,i}$ is not a singular measure. The proof is based on
Theorem~\ref{thm:Jurdjevic-2}.

For the $S$-valued process $A$ we denote by $I_t(A)$ the sequence of states visited by $A$ between $0$
and $t$.
For any $m\in\mathbb{N}$ and any sequence $\mathbf{i}\in S^m$, we can introduce an auxiliary measure $\mathsf{Q}_{\xi,i,\mathbf{i}}$ on $M$ by
\[
\mathsf{Q}_{\xi,i,\mathbf{i}}(B)=\int_{R_{+}} e^{-t}\,\mathsf{P}_{\xi,i}\{X_t\in B\ \text{and}\ I_t(A)=\mathbf{i}\}\,  dt,\quad B\in\mathcal{B}(M).
\]
Since
\begin{equation}
\label{eq:Qq-via-I_t}
\mathsf{Q}_{\xi,i}(B \times \{j\})=\sum_{m}\sum_{\mathbf{i}=(i,i_2,\ldots,i_{m-1},j)\in S^m}\mathsf{Q}_{\xi,i,\mathbf{i}}(B),
\end{equation}
it is sufficient to find $\mathbf{i}=(i_1,\ldots,i_m)$ with $i_1=i$ such that $\mathsf{Q}_{\xi,i,\mathbf{i}}(M)>0$ and
\[
\overline \mathsf{Q}_{\xi,i,\mathbf{i}}(\cdot)= \frac{\mathsf{Q}_{\xi,i,\mathbf{i}}(\cdot)}{\mathsf{Q}_{\xi,i,\mathbf{i}}(M)}
\]
 is a nonsingular probability measure. To apply Lemma~\ref{lem:pushforward_full_rank}, we need to represent
$\overline \mathsf{Q}_{\xi,i,\mathbf{i}}$ as the pushforward of a measure, equivalent to Lebesgue measure, under a smooth map with a
nonempty set of regular points.

Since Condition~B holds at $\xi$, Theorem~\ref{thm:Jurdjevic-2} yields an integer $m>n$
and a sequence $\mathbf{i}=(i_1,i_2,\ldots,i_{m+1})$ with $i_1=i$, such that the function $F_\mathbf{i}:\mathbb{R}_{+}^{m+1} \to
M$ defined by
\begin{equation*}
F_\mathbf{i}(\mathbf{t})= \Phi_{\mathbf{i}}(\mathbf{t},\xi)
\end{equation*}
has a regular point.
For this $\mathbf{i}$ provided by Theorem~\ref{thm:Jurdjevic-2}, $\overline \mathsf{Q}_{\xi,i,\mathbf{i}}$ is
the distribution of $\Phi_{{\bf i}} \left(T_1,\ldots,T_m,T-\sum_{j=1}^{m} T_j, \xi \right)$ conditioned on
\begin{equation}
R= \biggl\{\sum_{j=1}^{m} T_j <T \leq \sum_{j=1}^{m+1} T_j \biggr\},
\label{eq:event_R_2}
\end{equation}
where $T_1,\ldots,T_{m+1}$, and $T$ are independent random variables
exponentially distributed with parameters
$\lambda_{i_1},\ldots,\lambda_{i_{m+1}}$, and $1$, respectively.

Since the joint distribution of $T_1,\ldots,T_{m+1},T$ is equivalent to Lebesgue measure and since event $R$ has positive probability,
the distribution $\mu$ of $T_1,\ldots,T_m,T$ conditioned on $R$
induces a measure on
\[
\Delta= \biggl\{(t_1,\ldots,t_m,t) \in \mathbb{R}_{+}^{m+1}: \quad \sum_{j=1}^{m} t_j < t \biggr\}
\]
that is equivalent to Lebesgue measure. The regularity of $F_\mathbf{i}$ guaranteed by Theorem~\ref{thm:Jurdjevic-2} implies
that
the function $f_\mathbf{i}:\Delta\to M$ defined
by
\begin{equation*}
 f_\mathbf{i}(t_1,\ldots,t_m,t)= F_\mathbf{i}\biggl(t_1,\ldots,t_m,t-\sum_{j=1}^{m} t_j \biggr)
\end{equation*}
has a nonempty open set of regular points in~$\Delta$, and the proof is completed by an application of
Lemma~\ref{lem:pushforward_full_rank}, since $\overline \mathsf{Q}_{\xi,i,\mathbf{i}}$ is the pushforward of
$\mu$ under $f_\mathbf{i}$.

\section{Absolute continuity of ergodic invariant measures}\label{sec:AC-of-ergodic-measures}

According to the Ergodic Decomposition Theorem, all invariant measures for a Markov semigroup can be
represented as
convex combinations of ergodic ones (see, e.g., \cite[Theorem~5.7]{Hairer:ergodicity-lectures}). We will use this to derive Theorem~\ref{thm:uniqueness} from absolute continuity of ergodic invariant distributions.

To define ergodicity, we need to recall the notion of $\mu$-invariant sets. Let $\mu$ be an invariant measure for the
Markov semigroup $(\mathsf{P}^t)$. We say that a set $A\in\mathcal{B}(M)
\otimes \mathcal{P}(S)$ is $\mu$-invariant if for every $t\ge 0$, $\mathsf{P}^t_{\xi,i}(A)=1$ for $\mu$-almost every
$(\xi,i)\in A$. An invariant measure $\mu$ is called ergodic if for every $\mu$-invariant set $A$, either  $\mu(A)=1$ or
$\mu(A)=0$.

The following is a basic result on systems with Markov switchings that does not use Conditions A or B.

\begin{theorem}\label{thm:AC-or-S}
If $\mu$ is $(\mathsf{P}^t)$-invariant and ergodic then it is either absolutely continuous or singular.
\end{theorem}
\bpf
Consider the Lebesgue decomposition $\mu=\mu_{ac}+\mu_{s}$, where $\mu_{ac}$ is absolutely
continuous and $\mu_{s}$ is singular with respect to Lebesgue measure.  Let us show that both
$\mu_{ac}$ and $\mu_{s}$ are invariant.

For any $t>0$, using the
invariance of $\mu$, we can write
\begin{align}
\mu_{ac}+\mu_{s} &= \mu=\mu \mathsf{P}^t
\label{eq:decomposition_in_ac_and_s}
=
\mu_{ac} \mathsf{P}^t+\mu_s\mathsf{P}^t
=\sum_{j=1}^{k} \nu_j  + \mu_s\mathsf{P}^t,
\end{align}
where
\begin{equation}
\label{eq:nu_j}
\nu_{j}(\cdot)= \int_M \mathsf{P}^t_{\xi,j} (\cdot)\mu_{ac}(d\xi\times\{j\}),\quad j\in S.
\end{equation}

We claim that the measures $\nu_j$, $j\in S$,
are absolutely continuous. To see this we
check that for any sequence $\mathbf{i}=(i_1,\ldots,i_{m+1})$ with $i_1=j$, the measure $\nu_\mathbf{i}$ defined by
\begin{align}\notag
 \nu_{\mathbf{i}}(E)&= \int_M \mathsf{P}_{\xi,j} (X_t\in E\, |C_\mathbf{i})\mu_{ac}(d\xi\times\{j\})\\
&=\int_M \mathsf{P}\biggl(\Phi_{{\bf i}}\biggl(T_1,\ldots,T_m,t-\sum_{l=1}^{m} T_l,\xi \biggr)\in
E\, \biggr|\, R\biggr)\mu_{ac}(d\xi\times\{j\})\label{eq:one_of_continuous_components_disintegrated}
\end{align}
 is absolutely continuous (here we use the notation introduced in Section~\ref{sec:AC-component-1-proof}). Suppose
$\lambda^M(E)=0$. For fixed $T_1,\ldots,T_m,T_{m+1}$,  the map $\Phi_\mathbf{i}$ is a diffeomorphism in $\xi$. Therefore, on
event $R$ introduced in~\eqref{eq:event_R}, we have
\[
 \mu_{ac}\biggl(\xi \times \{j\}:\ \Phi_{{\bf i}}\biggl(T_1,\ldots,T_m,t-\sum_{l=1}^{m} T_l,\xi \biggr)\in
E  \biggr)=0,
\]
and $\nu_\mathbf{i}(E)=0$ follows from disintegrating the right side
of~\eqref{eq:one_of_continuous_components_disintegrated} and changing the order of integration.

Now, using~\eqref{eq:decomposition_in_ac_and_s} and the absolute continuity of $\nu_j$, $j\in S$, we can write
\begin{equation}
\label{eq:decomposition_of_ac_part}
\mu_{ac}=
\sum_{j=1}^{k} \nu_j  + (\mu_s\mathsf{P}^t)_{ac}\ .
\end{equation}
Since  $\mathsf{P}^t_{\xi,j}(M\times S)=1$ for all $\xi$ and $j$, \eqref{eq:nu_j} implies
$\sum_{j=1}^{k} \nu_j(M\times S)= \mu_{ac}(M\times S)$. Therefore, applying~\eqref{eq:decomposition_of_ac_part}
to $M \times S$, we obtain that the absolutely continuous
component of the measure $\mu_s\mathsf{P}^t$ is zero. In other
words, $\mu_s\mathsf{P}^t$ is singular, and from~\eqref{eq:decomposition_in_ac_and_s} and the absolute
continuity of $\nu_j$, $j\in S$, we obtain
\begin{equation}
\label{eq:invariance_of_singular_component}
\mu_s = \mu_s\mathsf{P}^t.
\end{equation}
In other words, $\mu_s$ is invariant for $(\mathsf{P}^t)$. It follows from~\eqref{eq:decomposition_in_ac_and_s}
that $\mu_{ac}$ is also invariant. Since $\mu$ is ergodic, it cannot be represented as a sum of two nontrivial
invariant measures. This means that either $\mu=\mu_{ac}$ or $\mu=\mu_s$.
{{\hfill $\Box$ \smallskip}}

We endow the state space $S$ with the discrete topology and recall that a point $(\xi,i)\in M\times S$ is
contained in the support of a measure if and
only if the measure of every open neighborhood of $(\xi,i)$ is positive.

\begin{theorem}\label{thm:absolute-continuity-for-ergodic}
Let $\mu$ be an ergodic invariant measure for $(\mathsf{P}^t)$. Assume that the support of $\mu$ contains a point
$(\eta,i)$ such that Condition~B holds at~$\eta$.
Then, $\mu$ is absolutely continuous
with
respect to Lebesgue measure on $M\times S$.
\end{theorem}

We will need several auxiliary statements.

\begin{lemma} \label{lem:support}
Let $\nu$ be a finite Borel measure on $M \times S$ with support $K$. If $U$ is any open set in $M \times
S$ whose intersection with $K$ is nonempty, we have
\begin{equation*}
\nu(U \cap K)>0.
\end{equation*}
\end{lemma}

\bpf
Assume that $\nu(U \cap K)=0$. The complement of the support of $K$ has measure zero. Therefore
\begin{equation*}
\nu(U) = \nu(U \cap K) + \nu(U \cap K^{c})=0.
\end{equation*}
Thus, $U^{c}$ is a closed subset of $M \times S$ whose complement has measure zero. From the
definition of the support, we obtain $K \subset U^{c}$. But then, $U \cap K$ must be empty, a contradiction.
{{\hfill $\Box$ \smallskip}}

\begin{lemma} \label{lem:open-starting-points}
Fix a time $t>0$, an open subset $E$ of $M$ and an index $j \in S$. The set $H$ of points $(\xi,i) \in M \times
S$ satisfying $\mathsf{P}^t_{\xi,i}(E\times\{j\})>0$ is open.
\end{lemma}

\bpf We will use the notation introduced in Section~\ref{sec:AC-component-1-proof}.

Let us assume that $H$ is not open and there is a sequence $((\xi_p,i))_{p\in\mathbb{N}}$ in $H^c$
converging to a point $(\xi,i)\in H$.

Since $(\xi,i)$ is an element of $H$, there exists an admissible sequence \\ $\mathbf{i}=(i,i_2,...,i_m,j)$ for which
$\mathsf{P}_{\xi,i}(X_t\in E|C_\mathbf{i})>0$.

On the other hand, we have $\mathsf{P}_{\xi_p,i}(X_t\in E|C_\mathbf{i})=0$ for all $p\in\mathbb{N}$. Therefore,
for each $p$, the set $B(\xi_p)$
has zero Lebesgue measure, where
\[
B(\eta)=\biggl\{\mathbf{t}\in\Delta_{t,m}:\, \Phi_{{\bf i}}\biggl(t_1,\ldots,t_m,t-\sum_{j=1}^{m} t_j,\eta \biggr)\in E
\biggr\},\quad \eta\in M.
\]

If $\mathbf{t}\in B(\xi_p)^c$ for all $p$,
then $\Phi_{\mathbf{i}}(\mathbf{t},\xi)\in E^c$ since $\Phi_{\mathbf{i}}(\mathbf{t},\cdot)$ is a diffeomorphism and $E^c$ is a closed set.
Hence,
the Lebesgue measure of $B(\xi)$ is also zero, so
$\mathsf{P}_{\xi,i}(X_t\in E|C_\mathbf{i})=0$, which contradicts our assumption and finishes the proof.
{{\hfill $\Box$ \smallskip}}

\bpf[Proof of Theorem~\ref{thm:absolute-continuity-for-ergodic}]  According to Theorem~\ref{thm:AC-or-S} we need to show
that $\mu$ is not singular. If $\mu$ is singular,
it is entirely supported on a zero Lebesgue measure set $G\subset M\times S$, so
$\mu(G^c)=0$. Since $\mu$ is $(\mathsf{P}^t)$-invariant, it is also $\mathsf{Q}$-invariant. Therefore,
$\mu(G^c)=\mu \mathsf{Q}(G^c)$, and we see that $\mu(V)=0$ where
\[
 V=\{(\xi,j)\in M \times S:\ \mathsf{Q}_{\xi,j} (G^{c})>0\}.
\]
Let $U$ be the set of points $\xi \in M$ where Condition~B holds.
Due to Theorem~\ref{thm:AC-component-2}, $U\times S \subset V$, and
we conclude that $\mu(U\times S)=0$.

Recall that $U$ is an open subset of $M$, and $(U\times S)\cap \mathop{\rm supp} \mu\ne\emptyset$ by assumption.
Lemma~\ref{lem:support} implies that  $\mu((U\times S)\cap \mathop{\rm supp} \mu)>0$.  The contradiction with $\mu(U\times
S)=0$ completes the proof.
{{\hfill $\Box$ \smallskip}}

Of course, if one replaces Condition~B in Theorem~\ref{thm:absolute-continuity-for-ergodic} with
the stronger Condition~A the resulting statement holds automatically, but one can give a proof that does not involve
the resolvent $Q$:

\begin{theorem}\label{thm:absolute-continuity-for-ergodic-under-condition-A}
Let $\mu$ be an ergodic invariant measure for $(\mathsf{P}^t)$. Assume that the support of $\mu$ contains a point
$(\eta,i)$ such that Condition~A holds at~$\eta$.
Then, $\mu$ is absolutely continuous
with
respect to the Lebesgue measure on $M\times S$.
\end{theorem}

\bpf  According to Theorem~\ref{thm:AC-or-S} we need to show that $\mu$ is not singular. If $\mu$ is singular,
it is entirely supported on a zero Lebesgue measure set $G\subset M\times S$, so
$\mu(G^c)=0$. Since $\mu(G^c)=\mu \mathsf{P}^t(G^c)$, we see that $\mu(V)=0$ where
\[
 V=\{(\xi,j)\in M \times S:\ \mathsf{P}^t_{\xi,j} (G^{c})>0\}.
\]
Let $U$ be the set of points $\xi \in M$ where Condition~A holds.
Due to Theorem~\ref{thm:AC-component-1}, $U\times S \subset V$, and
 we conclude that $\mu(U\times S)=0$.

Recall that $U$ is an open subset of $M$, and $(U\times S)\cap \mathop{\rm supp} \mu\ne\emptyset$ by assumption.
Lemma~\ref{lem:support} implies that  $\mu((U\times S)\cap \mathop{\rm supp} \mu)>0$.  The contradiction with $\mu(U\times
S)=0$ completes the proof.
{{\hfill $\Box$ \smallskip}}

\section{Proof of Theorem~\ref{thm:uniqueness}} \label{sec:uniqueness-proof}

First, we establish two properties of the set $E= L\cap U$, where
$U$ is the open set of points satisfying Condition B.
\begin{lemma} \label{lem:non-empty-interior}
The set $E$ has nonempty interior.
\end{lemma}

\bpf  By assumption,  $\xi\in E$ , so $U\ne\emptyset$ and $L(\xi) \cap U\ne \emptyset$ by continuity of the vector fields in $D$. Since $\xi\in U$, Theorem~\ref{thm:Jurdjevic-2} implies that $L(\xi)$
has nonempty interior that is dense in $L(\xi)$. Therefore, the set
\begin{equation*}
V=L(\xi)^{\circ} \cap U
\end{equation*}
is nonempty and open. Clearly, $V\subset U$, and it remains to prove that $L(\xi)^{\circ}\subset L$.
It is sufficient to show that  $W\cap L(\eta)\ne\emptyset$ for any nonempty open subset $W$ of $L(\xi)$ and any $\eta \in M$.

Let us take any $\zeta\in W$. Since $\zeta$ is $D$-reachable from $\xi$, there exist ${\bf i}$ and~${\bf t}$ such that
\begin{equation*}
\zeta = \Phi_{{\bf i}}({\bf t},\xi).
\end{equation*}
The mapping $h$ defined by $h(x)=\Phi_{{\bf i}}({\bf t},x), x\in M,$ is a diffeomorphism on $M$. Therefore, $h^{-1}(W)$ is an open neighborhood of $\xi$.
Since $\xi \in E\subset \overline {L(\eta)}$, there is a point $\rho \in h^{-1}(W)$, $D$-reachable from $\eta$. Consequently,
$h(\rho)\in W$ is also $D$-reachable from $\eta$.
{{\hfill $\Box$ \smallskip}}

As an immediate corollary of Lemma~\ref{lem:non-empty-interior}, the set $L$ has nonempty interior.
\begin{lemma} \label{lem:positive-measure}
Suppose $\mu$ is an invariant measure for  $(\mathsf{P}^t)$. If $G$ is a nonempty open subset of~$L$ and $j\in
S$, then $\mu(G\times\{j\})>0$.
\end{lemma}

\bpf Let us assume that $\mu(G\times\{j\})=0$. Since
$\mu$ is $(\mathsf{P}^t)$-invariant, it is also $\mathsf{Q}$-invariant, and we have
\begin{equation*}
0 = \mu(G\times\{j\}) = \sum_{i=1}^{k} \int_M \mathsf{Q}_{\eta,i}(G\times\{j\}) \mu(d\eta\times\{i\}).
\end{equation*}
For all $i\in S$ and $\mu(\cdot \times\{i\})$-almost every $\eta\in M$, we thus obtain
\begin{equation}
\label{eq:Q-0}
\mathsf{Q}_{\eta,i}(G\times\{j\}) = 0.
\end{equation}
Let us choose $\eta$ such that~\eqref{eq:Q-0} holds true.

By assumption, we have $G\subset L\subset\overline{L(\eta)}$. Since $G$ is open,  $G \cap L(\eta)\ne
\emptyset$. So there exist  a sequence $\mathbf{i}=(i,i_2,\ldots,i_m,j)$ and an interswitching time vector
$\mathbf{t}=(t_1,\ldots,t_m,t_{m+1})$ such that  $\Phi_{{\bf i}}({\bf t}, \eta)\in G$. By continuity of $\Phi_{\bf i}$ there
is a neighborhood $W$ of $\mathbf{t}$ in $\mathbb{R}^{m+1}_+$
such that $\Phi_{{\bf i}}(\mathbf{s}, \eta)\in G$ for all $\mathbf{s}\in W$.  Denoting $s=s_1+\ldots+s_{m+1}$ and using the
representation of $\mathsf{P}^s_{\eta,i}(\cdot|C_{\mathbf{i}})$ via exponentially distributed times
that we used in the proof of Theorem~\ref{thm:AC-component-1}, we conclude that   $\mathsf{P}^s_{\eta,i}(G \times \{j\})>0$ for
$s$ sufficiently close to $t=t_1+\ldots+t_{m+1}$. Therefore, $\mathsf{Q}_{\eta,i}(G \times \{j\})>0$ contradicting \eqref{eq:Q-0}.
{{\hfill $\Box$ \smallskip}}

\bigskip

\bpf[Proof of Theorem~\ref{thm:uniqueness}]
Any invariant measure can be written as a convex combination of ergodic invariant measures. Therefore, it suffices to
show absolute continuity and uniqueness of an ergodic invariant measure.

Let us begin by deriving absolute continuity. If $\mu$ is an
ergodic invariant measure that satisfies the assumptions of Theorem~\ref{thm:uniqueness} then,
due to Theorem~\ref{thm:absolute-continuity-for-ergodic}, it suffices to show that
$L \subset \mathop{\rm supp} \mu$. Let $\xi \in L$, and let $U$ be a neighborhood of $\xi$ in $M$, $j \in S$. By
Lemma~\ref{lem:positive-measure}, we have $\mu(U \times \{j\})>0$, hence $\xi \in \mathop{\rm supp} \mu$.

In order to prove uniqueness of the ergodic invariant measure, let us assume that $\mu_1$ and $\mu_2$ are two distinct ergodic invariant probability measures. Birkhoff's ergodic
theorem then implies that $\mu_1$ and $\mu_2$ are mutually singular. Hence, the set $M \times S$ can be partitioned into
two disjoint subsets $H_1$ and $H_2$ with
$\mu_1(H_2) = \mu_2(H_1) = 0.$
The two sets can be represented as
\[H_\alpha=\bigcup_{j=1}^{k} M_{\alpha,j} \times \{j\},\quad \alpha=1,2,\]
for some measurable sets $M_{\alpha,j}$, $j\in S$, $\alpha=1,2$. For all $\alpha$ and $j$,
\begin{equation*}
\mu_\alpha( M_{\alpha,j} \times \{j\})=\mu_\alpha( M \times \{j\})>0,
\end{equation*}
since the left side is a stationary distribution for the Markov chain on $S$ and by our assumptions, transitions
between all states happen with positive probability.

Fix a $j$ in $S$. By virtue of Lemma~\ref{lem:non-empty-interior}, the set $E^{\circ}$ is nonempty.
According to Lemma~\ref{lem:positive-measure}, for all $j \in S$ we have $\mu_1(E^\circ\times\{j\})>0$.
 Since $\mu_1(M_{2,j}\times\{j\}) =0$, we deduce that $\mu_1(E_1\times\{j\}) > 0$, where $E_1=E^\circ \cap
M_{1,j}$.

The measure $\mu_1$ is $(\mathsf{P}^t)$-invariant, hence it is also $\mathsf{Q}$-invariant, and we have
\begin{equation}
\label{eq:invariance-for-M_2_j}
0=\mu_1(M_{2,j}\times\{j\}) \geq \int_{E_1} \mathsf{Q}_{\eta,j}(M_{2,j}\times \{j\}) \mu_1(d\eta\times\{j\}).
\end{equation}
Since $\mu_1(E_1\times\{j\})>0$, it suffices to show that $\mathsf{Q}_{\eta,j}(M_{2,j}\times\{j\})>0$  for all $\eta \in
E_1$, to obtain a contradiction with~\eqref{eq:invariance-for-M_2_j}.

Since $\eta$ satisfies Condition~B,
Theorem~\ref{thm:Jurdjevic-2} guarantees that there exist an integer $m>n$ and a vector ${\bf i} =
(j,i_2,\ldots,i_m,j)$ such
that the function $f:\mathbb{R}^{m+1}_+ \to M$ defined by
\begin{equation*}
 f(\bf t) =\Phi_{\bf i}({\bf t},\eta)
\label{eq:F_ibf}
\end{equation*}
has an open set $O$ of regular points such that for all $t>0$,
\begin{equation*}
\{\mathbf{t}=(t_1,\ldots,t_{m+1})\in O:\ t_1+\ldots+t_{m+1}<t\}\ne \emptyset.
\end{equation*}
Therefore the map $F$ defined by
\[
 F(t_1,\ldots,t_{m+1},t)= f\biggl(t_1,\ldots,t_m,t-\sum_{l=1}^{m}t_l\biggr)
\]
on
\[
 \Delta=\biggl\{(t_1,\ldots,t_{m+1},t)\in\mathbb{R}_{+}^{m+2}:\ \sum_{l=1}^{m}t_l<t<\sum_{l=1}^{m+1}t_l\biggr\}
\]
has an open set $V\subset\Delta$ of regular points such that
\begin{equation}
\{\mathbf{t}=(t_1,\ldots,t_{m+1},t)\in V:t<s\}\ne \emptyset,\quad s>0.
\label{eq:0_is_limiting_point}
\end{equation}

Using the representation of $\mathsf{Q}$ via~\eqref{eq:Qq-via-I_t} and the family of exponentially distributed times
$T_1,\ldots,T_{m+1},T$, we obtain that it is sufficient to prove that
\begin{equation}
\mathsf{P}\left\{F(T_1,\ldots,T_{m+1},T) \in M_{2,j}|\ R \right\} > 0,
\label{eq:sufficient_for_contradiction}
\end{equation}
where $R$ was introduced in~\eqref{eq:event_R_2}.

 Since $E^\circ$ is an open set containing
$\eta$, and $F(V)$ is an open set such that $\eta\in \overline{F(V)}$ (due
to~\eqref{eq:0_is_limiting_point} and continuity of $F$ at $0$), we obtain that
$G= E^{\circ} \cap F(V)$ is also a nonempty open set.

Let us choose a vector ${\bf r}\in V$ such that $F({\bf r}) \in E^{\circ}$. Since $\mathbf{r}$ is a regular point for
$F$, we see that
for an arbitrary choice of local smooth coordinates around $\mathbf{r}$,
there are $n$ independent columns of the matrix $DF(\mathbf{s})$ for $\mathbf{s}$
in a small neighborhood of $\mathbf{r}$. Without loss of generality we
can assume that these are the first $n$ columns.
Then the map $\rho:\mathbb{R}^{m+2}\to M\times\mathbb{R}^{m+2-n}$ defined by
\[
 \rho(s_1,\ldots,s_{m+1},s)=(F(s_1,\ldots,s_{m+1},s),s_{n+1},\ldots ,s_{m+1},s)
\]
has nonzero Jacobian in that neighborhood. So we can choose an open set $W_V$ containing~$\mathbf{r}$ so that $\rho$ is a
diffeomorphism between $W_V$ and
$W_G\times W_{n-m-2}$, where $W_G\subset G$ and $W_{m+2-n}\subset \mathbb{R}_{+}^{m+2-n}$ are some open sets.

The set $W_G$ is an open subset of $L$. It is also not empty since it contains~$F({\bf r})$.
Lemma~\ref{lem:positive-measure} implies that $\mu_2( W_G\times\{j\})>0$.  Since
$\mu_2(M_{2,j}^c\times\{j\})=0$, we conclude that
$\mu_2(J\times\{j\})>0$ where $J=M_{2,j} \cap W_G.$ Since $\mu_2$ is an ergodic measure, it is absolutely continuous, so
\begin{equation}
\label{eq:J-Lebesgue-positive}
\lambda^M(J)>0.
\end{equation}

Since $J\subset M_{2,j}$, the desired inequality \eqref{eq:sufficient_for_contradiction} will follow from
\begin{equation}
\mathsf{P}\left\{F(T_1,\ldots,T_{m+1},T) \in J|\ R \right\} > 0.
\label{eq:sufficient_for_contradiction-2}
\end{equation}

Since the joint distribution of $T_1,\ldots,T_m,T_{m+1},T$ is equivalent to the Lebesgue measure on $\Delta$,
Lemma~\ref{lem:pushforward_for_nonzero_det} implies that $\rho(T_1,\ldots,T_{m+1},T)$ has positive density
almost everywhere in $W_G\times W_{m+2-n}$. Integrating over $W_{m+2-n}$, we see that \\ $F(T_1,\ldots,T_{m+1},T)$ has
positive density almost everywhere in $W_G$. Now~\eqref{eq:sufficient_for_contradiction-2} follows from
\eqref{eq:J-Lebesgue-positive}.{{\hfill $\Box$ \smallskip}}

\bigskip

Of course, Theorem~\ref{thm:uniqueness} remains true if one replaces Condition~B by the stronger Condition~A.
However, under that condition one can prove this result without referring to the resolvent $\mathsf{Q}$. Namely, one can
use the regularity of transition probabilities established in Theorem~\ref{thm:AC-component-1} (which is stronger than
the regularity established in Theorem~\ref{thm:AC-component-2}),
and invoke Theorems~\ref{thm:Jurdjevic-1} and~\ref{thm:absolute-continuity-for-ergodic-under-condition-A} instead of
Theorems~\ref{thm:Jurdjevic-2} and~\ref{thm:absolute-continuity-for-ergodic}.

\section{Examples} \label{sec:examples}

In this section, we apply Theorem~\ref{thm:uniqueness} to two concrete switching systems. In the first example, we have a closer look at the system on the $n$-dimensional torus $\mathbb{T}^n=\mathbb{R}^n/\mathbb{Z}^n$ that was introduced in Section~\ref{sec:absolute-continuity-invariant}. In the second example, we switch between two Lorenz vector fields with different parameter sets. For both systems, uniqueness of the invariant measure is derived from Theorem~\ref{thm:uniqueness}. For the system on $\mathbb{T}^n$, we point out the invariant measure explicitly.

\bigskip

Let $M$ be the $n$-dimensional torus $\mathbb{T}^n$, and let $D=\{u_1,\ldots,u_n\}$ be the standard basis of $\mathbb{R}^{n}$.
 We assume for simplicity that the parameter $\lambda$ of the exponential time between any two switches is independent
of the current state, and that we have a uniform probability of switching between any two states. In
Section~\ref{sec:absolute-continuity-invariant} we implicitly argued that Condition~A does not hold for this system: If
Condition~A was satisfied at some point $\xi \in \mathbb{T}^n$, the transition probability measures $\mathsf{P}_{\xi,i}^t$ would not be
singular with respect to Lebesgue measure, according to Theorem~\ref{thm:AC-component-1}. However, as pointed out in
Section~\ref{sec:absolute-continuity-invariant}, the measures $\mathsf{P}_{\xi,i}^t$ are purely singular.

It is also instructive to show directly why Condition~A does not hold. As all the vector fields in $D$ are constant, the derived algebra $\mathcal{I}'(D)$ contains only the zero vector field. Thus, for any $\xi \in \mathbb{T}^n$,
\begin{equation*}
 \mathcal{I}_{0}(D)(\xi) = \biggl\{\sum_{i=1}^{n} \lambda_{i} u_{i}:\ \sum_{i=1}^{n} \lambda_{i} = 0\biggr\}.
\end{equation*}
Due to the constraint $\sum_{i=1}^{n} \lambda_{i} = 0$, the algebra $\mathcal{I}_{0}(D)(\xi)$ does not have full
dimension, so Condition~A is violated at every point in $\mathbb{T}^n$.

On the other hand, Condition~B is clearly satisfied at any point $\xi \in \mathbb{T}^n$, as the standard basis of $\mathbb{R}^n$ applied to $\xi$ yields a full-dimensional set of vectors in the tangent space. Also note that any point in $\mathbb{T}^n$ is $D$-reachable from any other point. Therefore, Theorem~\ref{thm:uniqueness} guarantees that the associated Markov semigroup has a unique invariant measure, provided that such a measure exists. In this elementary example, it is possible to point out the invariant measure explicitly. For Borel sets $E \subset \mathbb{T}^n$ and states $i \in S$, it is given by
\begin{equation*}
 \mu(E \times \{i\}) = \frac{1}{n} \cdot \lambda(E).
\end{equation*}
Here, $\lambda$ denotes Lebesgue measure on $\mathbb{T}^n$.

\bigskip

The second example provides a situation where (i) the number of vector fields in~$D$ is less than the
dimension of the manifold $M$, and (ii) each individual vector field in~$D$ gives rise to dynamics with a
strange attractor and no absolutely continuous invariant measures, but (iii) the switched system has a unique invariant
measure and it is absolutely continuous.

Namely, we consider switching between two Lorenz vector fields with different parameter values. A Lorenz
vector field is a vector field defined in $\mathbb{R}^3$, of the form
\begin{equation*}
 u(x,y,z) = \begin{pmatrix}
             \sigma \cdot (y-x) \\
             rx-y-xz \\
             xy-bz
            \end{pmatrix},
\end{equation*}
 where $\sigma$, $r$ and $b$ are physical parameters. Let the set $D$ contain exactly two Lorenz vector fields $u_1$ and
$u_2$ such that $u_1$ has Rayleigh number $r=28$ and $u_2$ has a Rayleigh number different from, but close to, $28$. We
assume for both vector fields that $\sigma=10$ and that $b=\tfrac{8}{3}$, which is the classical parameter choice for
the Lorenz system. In~\cite{Tucker:MR1701385}, Tucker shows that the Lorenz system with parameters $\sigma=10$, $r=28$
and $b=\tfrac{8}{3}$, corresponding to vector field $u_1$, admits a robust strange attractor $\Lambda$ as well as a
unique SRB-measure supported on $\Lambda$. Robustness implies that the dynamical structure of the system remains intact
under small parameter changes, so the dynamics induced by $u_2$ share these features if $r$ is sufficiently close to
$28$. Moreover, the SRB-measure on $\Lambda$ satisfies a dissipative ergodic theorem, see
e.g.~\cite[Section~5.1]{Bunimovich-Sinai:MR755521}. It follows that any point $\xi\in\Lambda$ is
$\{u_1\}$-approachable (and thus $D$-approachable) from every point in a set $S_\xi\subset\mathbb{R}^3$ with zero Lebesgue
measure complement.

Assisted by a computer algebra system, we checked that Condition~A is satisfied for this system at any point in $\mathbb{R}^3$
that does not lie on the $z$-axis. Since the $z$-axis is invariant under the flows of both vector fields, we disregard
it and set $M$ to be $\mathbb{R}^3$ without points on the $z$-axis. With this provision, every point on the attractor $\Lambda$
is $D$-approachable from any point in $M$:

Consider a point $\xi \in \Lambda$ and a point $\eta \in M$. By Theorem~\ref{thm:Jurdjevic-1}, there is a nonempty open
set of $D$-reachable points from $\eta$ (recall that Condition~A holds at any point in $M$). And since this open set has
positive Lebesgue measure, it contains a point belonging to~$S_\xi$.
Hence, $\xi$ is $D$-approachable from $\eta$. As in the first example, uniqueness and absolute continuity of an invariant measure
follow from Theorem~\ref{thm:uniqueness}.

\bibliographystyle{alpha}


\section{Introduction}

Alzheimer's disease (AD) is the most common form of dementia, usually associated with the elderly population (over 65 years of age). AD had a worldwide prevalence of around 33.9 million cases by 2011, with predictions suggesting an increase to about 100 million by 2050
\citep{barnes2011}. Recently, the view on AD diagnosis has shifted towards a more dynamic process in which clinical and pathological markers evolve gradually before diagnostic criteria are met. The AD Assessment Scale-cognition sub-scale (ADAS-Cog)~\citep{mohs1997development} is the most widely used general cognitive measure in clinical trials of AD~\citep{skinner2012alzheimer}. While it was developed as an outcome measure for dementia interventions, its primary purpose was to be an index of global cognition in response to antidementia therapies. The ADAS-Cog assesses multiple cognitive domains including memory, language, praxis, and orientation~\citep{skinner2012alzheimer}. Because ADAS-Cog has proven important for target clinical assessments, in this paper we focus on a machine learning method that can successfully forecast the future values of this score for target subjects. Specifically, we use the modified ADAS-Cog 13-item scale~\citep{mohs1997development}, which is scored on a 85 point scale, where higher scores indicate greater severity.

\begin{figure}[t]
\centering
\caption{{\small {\bf Personalized GP Experts (pGPE):} The population model is first trained using all visits data of $N$ training subjects ($x^{TR}$ are the input features and $y^{TR}$ are the corresponding ADAS-Cog13 scores), where the time difference between two visits is 6 months. The model personalization to the target (previously unseen) subject ($N+1$) is then achieved by sequentially adapting the model predictions of the future ADAS-Cog13 scores $y_{t+1:t+4}$ (using the posterior distribution of GPs - $f_{GP}$), informed by the visits data up to time step $t$, in the personalized GP (pGP) model. We also train the subject-specific model - target GP (tGP) - using only the past data of the target subject. The shaded fields in the output vector represent the time points for which we aim to predict ADAS-Cog13 scores. The proposed pGPE performs a regression-based weighting of the experts' pGP and tGP using meta-weights ($\alpha$), estimated using a GP regression model (GP). In this way, the optimal weights are trained for each subject and his/her future visit, leading to large improvements over the individual experts (pGP and tGP) in the target task.}}
\label{overview}
\includegraphics[scale=.3]{overview_fig_v12.png}
\end{figure}
One of the main challenges in the clinical assessment of subjects at risk of developing AD is the ability to accurately predict subjects' future cognitive scores. Such predictions can play an important role in subject selection during the design of AD clinical trials. For this, an automated approach that could forecast future cognitive scores, such as ADAS-Cog13, would be of great value during assessment procedures, and could potentially improve clinical trial design and early detection of at-risk subjects. For example, out of hundreds of clinical trials, costing billions of dollars, fewer than $1\%$ have proceeded to the regulatory approval stage and none have managed to prove a disease-modifying effect \citep{tadpole2017,cummings2006}.
Previously, it has been suggested that the low success rate of pharmaceutical AD clinical trials could be due to the inclusion of study populations that were too heterogeneous \citep{falahati2014}.
More successes depend on an improved ability to accurately identify subjects at early ages of the disease where treatments are most likely to be effective. Thus, developing models with improved ability to automatically predict subjects' future AD-related metrics indicating disease progression -- and to do so as early as possible, especially before the emergence of clinical symptoms -- is an important step towards this end. Furthermore, accurate prediction of symptom onset in the time window of 6 to 24 months is critical to participant selection and formation of clinical trials. Thus, having access to accurate future estimates of the progression of cognitive scores such as ADAS-Cog13 within this time frame is of great importance.

In this paper, data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI)~\citep{weiner2017} were used, and, specifically, the dataset processed for the TADPOLE Challenge \citep{tadpole2017}. These data are highly heterogeneous and multi-modal, and include imaging (MRI, PET), cognitive scores, CSF biomarkers, genetics, and demographics (e.g. age, gender, race). Although the heterogeneous nature of this dataset lends itself to building powerful, informative multi-modal models, the dataset itself is very sparse, with different combinations of features missing for different subjects. Partial records are also missing for roughly $80\%$ of the subjects \citep{campos2015}. Moreover, given the wide variability in available data per subject, inherent per-person differences, and the slowly changing nature of the disease, accurate forecasting of cognitive decline and related measures of disease progression is a significant and difficult challenge. To tackle these challenges, we focus on machine learning models that can easily adapt to each subject when forecasting his/her cognitive scores. More specifically, we investigate the effects of the model personalization for the forecasting task using the framework of Gaussian Processes (GP)~\citep{rasmussen2006gaussian}. This non-parametric probabilistic framework offers great modeling flexibility for not only predicting future cognitive scores but also uncertainty in their predictions. This, in turn, provides a principled approach for model adaptation to target subjects via posterior distribution of GP~\citep{peterson2017personalized}. Moreover, GP are well-suited for forecasting tasks where data are noisy time series with many missing values~\citep{futoma2018gaussian}, as in ADNI.

To this end, we used a cohort of subjects from the ADNI dataset (see Sec.~\ref{exp}) to predict future ADAS-Cog13 scores of target subjects using data from each subject’s previous visits. A subject's visit is defined as data collected at a single time point (a subject visit) during ADNI. We start by training a population-level model using multi-modal data of previously-seen (source) subjects using the base GP regression (sGP). Then, this model is adapted sequentially over time to a new (target) subject using the notion of domain adaptive GPs~\citep{liu2015bayesian,eleftheriadis2017}. We extend this personalization approach, denoted as pGP~\citep{peterson2017personalized}, for the forecasting task of predicting the ADAS-Cog13 scores at 6, 12, 18 and 24 months in the future. We also compare this approach with the subject-specific GP model, trained only on the data of the target subject (tGP). While we show that these models achieve good performance when selectively applied to the forecasting task (one performs better than the other for certain subjects/visits), their individual (average) performance is suboptimal.
To this end, we propose a novel weighting scheme for the GP experts based on the notion of meta-learning~\citep{vanschoren2018meta}. The main idea of meta-learning is to observe how different machine learning approaches perform on a wide range of learning tasks, and then learn from this experience, or meta-data, to learn new tasks more effectively. Following this approach, we designed a regression-based weighting of these expert models in the proposed personalized GP experts (pGPE) model, where the expert weights are optimized for each subject and his/her future visit using data-derived meta-features (see Sec.~\ref{pgpe}). Our results show that this newly introduced personalized weighting of the expert models leads to large improvements in accurately forecasting future ADAS-Cog13 scores for each target subject, while also outperforming traditional weighting schemes for expert models (e.g. by simply averaging the model predictions or using the GP-variance-based weighting), and the standard models for time-series forecasting tasks.

\section{Related Work}
We briefly review here the related work on forecasting of cognitive scores and clinical status for AD assessment, with the focus on the ADNI dataset. Most existing approaches, e.g.,~\citep{schmidt-richberg16,guerrero2016,gavidia2017}, focus on modeling subjects based on their clinical status (CS): cognitively normal (CN), mild cognitive impairment (MCI) and Alzheimer's Disease (AD). Additionally, the majority of these model biomarkers at the population level; for instance, estimating typical trajectories of markers over the full course of the disease to estimate current disease progress and progression rate \citep{schmidt-richberg16,guerrero2016}. Guerrero et al.~\citep{guerrero2016} used mixed effects modeling to derive global and individual biomarker trajectories for a training population, which was later used to instantiate subject-specific models for unseen subjects. Some of the modeling techniques \citep{guerrero2016,schmidt_richberg15_a,schmidt-richberg16} require cohorts with known disease onset and are prone to bias due to the uncertainty of the conversion time.

When it comes to forecasting clinical status, several authors attempted predicting target scores for longer time windows. For example, several recent works have explored the use of multiple, multi-modal predictors in combination with various machine learning techniques (e.g. SVMs, neural nets, GPs, etc.) to predict conversion from MCI to AD for various future time periods, e.g. from 1-5 years after baseline assessment~\citep{long2017,minhas2017}. Likewise, the BrainAGE framework has been proposed for predicting MCI-AD conversion within 3 years of follow-up \citep{gaser2013}. In addition, to predict conversion within time windows of up to 2 years (short-term converter) and 2-4 years (long-term converter), \citep{pereira2017} proposed a stepwise learning approach, where the learned model first predicts whether a subject converts to dementia, or remains stable, and then predicts the more likely progression window (short-term or long-term conversion). More recently, \citep{grassi2018a} evaluated different machine learning models in order to develop an algorithm for a 3-year prediction of conversion to AD in MCI and PreMCI subjects based only on non-invasive and effectively collectible predictors.

Most of these works attempted forecasting the changes in subjects' CS, which deals with a limited number of future outcomes (i.e. either binary or a three-class). By contrast, we aim to forecast ADAS-Cog13, defined on a more fine-grained scale (85 levels), which poses a more challenging machine learning problem. While recent work investigated forecasting of ADAS-Cog13~\citep{peterson2017personalized}, along with other cognitive scores from the TADPOLE challenge, using the GP framework, their forecasting horizon was limited as it only predicted one time step ahead (6 months). In this work, we attempt forecasting of ADAS-Cog13 up to 24 months ahead. While recent work in~\citep{utsumi2018pgp} has shown that a simple combination of personalized GP models leads to better prediction accuracy of the target cognitive scores, to the best of our knowledge, no prior work has proposed a principled approach for combining this expert knowledge. In this paper, we do so using the notion of meta learning to design an optimal weighting scheme for personalized GP models.

\section{Methodology}
\label{methods}
{\bf Notation.} We consider a supervised setting, where $\vect{X} =\{\vect{X}^{\bf(s)},\vect{X}^{\bf(t)}\}$
represent the input features to our model, and superscripts $\bf l=\{s,t\}$ refer to the data of source (i.e., training) $\bf (s)$  and target $\bf (t)$ subjects, respectively. Similarly, the ADAS-Cog13 scores we aim to forecast are stored in $\vect{Y} =\{\vect{Y}^{\bf (s)},\vect{Y}^{\bf(t)}\}$. More specifically, $\vect{X}^{\bf(l)} = \{\vect{x}^{\bf(l)}_{n}\}_{n=1}^{N_{\bf l}}$, where $N_{\bf s}$ and $N_{\bf t}$ are the numbers of the source and target subjects, respectively. Furthermore, $\vect{x}=[\vect{x}_1,\dots,\vect{x}_t,\dots,\vect{x}_{\vect{T}}]$ are the input features (predictors) for each time step of the subject's data\footnote{The subscript $t$ refers to time and is different from superscript $\bf(t)$ that refers to the target subject.}, and $\vect{T}\leq13$ is the maximum number of visits per subject over the period of 10 years, which varies largely per subject. Each $\vect{x}_t$ is a vector containing the features as $\vect{x}_t=[x_t^{m_1},\dots,x_t^{m_6},y_t]^T$, where $T$ is the transpose operation, and $m_{i=1,\dots,6}$ denotes the features from each data modality. We also add the current cognitive score $y_t\in\text{(0-85)}$ as a predictor since we found that it is a strong predictor of the future scores. The scores that we forecast are stored in $\vect{Y}^{\bf(l)} = \{\vect{y}^{\bf(l)}_{n}\}_{n=1}^{N_{\bf l}}$, and as $\vect{y}=[\vect{y}_1,\dots,\vect{y}_t,\dots,\vect{y}_{\vect{T}}]$. Here, each $\vect{y}_t$ contains the scores in the forecasting window of four steps ahead, and is given by $\vect{y}_t=[y_{t+1},\dots,y_{t+4}]^T$. Thus, the goal of the forecasting models proposed here is to learn an efficient mapping: $\vect{X}\rightarrow\vect{Y}$. As many subjects in ADNI missed certain visits and not all biomarkers were recorded at every visit, we fill in subjects' missing values using their nearest available past visit; however, no data of future visits are used.\footnote{To tackle this, more advanced approach based on auto-encoders can be used, e.g., see~\citep{campos2015}, and we leave this for the future work.}

\subsection{Population-level GPs}
We first build the population-level forecasting model using data from the source subjects $\{\vect{X}^{\bf(s)},\vect{Y}^{\bf(s)}\}$.
To this end, we use the GP framework~\citep{rasmussen2006gaussian} to train the following forecasting function:
\begin{equation}
 \vect{y}_{t}^{\bf (s)} = f(\vect{x}_{t}^{\bf (s)}) + \epsilon_{t}^{\bf (s)},
\end{equation}
where $\epsilon_t^{\bf(s)}\sim \mathcal{N}(0, \sigma^2_{\bf s})$ is i.i.d. additive Gaussian noise, which variance $\sigma^2_{\bf s}$ is estimated from the training data. In this framework, central to learning of the forecasting function $f(\cdot)$ is the GP prior that is placed over the function space, leading to the marginal likelihood of our data $p(\vect{Y}^{\bf(s)}|\vect{X}^{\bf(s)}) = \mathcal{N}(\vect{Y}^{\bf(s)}|\vect{0}, \vect{K}^{\bf(s)}+\sigma_{\bf s}^2\vect{I})$, where the elements of the kernel matrix are given by $\vect{K}^{\bf(s)} = \vect{k}(\vect{X}^{\bf(s)},\vect{X}^{\bf(s)})$ and $\vect{I}$ is the identity matrix. The choice of the kernel function $k(\cdot,\cdot)$ is at the heart of GPs as it encodes the relationships in our data. To this end, we use the standard radial basis function (RBF) kernel. Specifically, we investigate two types of the RBF kernel: isotropic (iso) and automatic relevance determination (ard)~\citep{rasmussen2006gaussian}. The latter assigns different weights (length-scales) to each feature (in our case, each data modality), effectively doing feature selection.

The kernel parameters $\vect{\theta}$ are optimized by minimizing the negative log-marginal likelihood: $-\log p(\vect{Y}^{\bf(s)}|\vect{X}^{\bf(s)}, \vect{\theta})$ using conjugate gradient method. Then, given the data from the visit at time $t$ of a {\it new} subject, $\vect{x}_\ast=\{\vect{x}_{t},\vect{y}_{t}\}$, the GP predictive distribution provides the mean and variance forecasts of the cognitive scores $\vect{y}_\ast$ as:
\begin{align}
\small
\label{post_mu}\vect{\mu_\ast^{\bf(s)}} &= \vect{k}_\ast^T (\vect{K}^{\bf(s)} + \sigma_{\bf s}^2\vect{I})^{-1}\vect{Y}^{\bf(s)} \\
\label{post_s}\vect{V_\ast^{\bf(s)}} &= k_{\ast\ast} -
\vect{k}_\ast^T (\vect{K}^{\bf(s)} + \sigma_{\bf s}^2\vect{I})^{-1}
\vect{k}_\ast,
\end{align}
where {\small $\vect{k}_\ast = k(\vect{X}^{\bf(s)}, \vect{x}_\ast)$} and {\small $k_{\ast\ast} = k(\vect{x}_\ast, \vect{x}_\ast)$}. We use the mean of this predictive distribution for the point estimate of target outputs, denoted as $\vect{\mu}_\ast^{\bf(s)}=\vect{\mu}_\ast^{\bf(s)}(\vect{x}_\ast)$ for notational convenience. Note that we use the shared covariance function for simultaneous forecasting of the four future scores in $\vect{y}_*$. Consequently, the model assigns the same variance ($\vect{V}_{*}$) to the forecasting window. We refer to this setting as the source GP (sGP).

\subsection{Personalized GPs (pGP and tGP)}\label{pgp_tgp}
We use the notion of domain adaptive GPs (DA-GP)~\citep{liu2015bayesian,eleftheriadis2017} to personalize the population GP model to target subjects. This is achieved by sequentially adapting the GP posterior for the test subject using the data of his/her past visits, to forecast the future ADAS-Cog13 scores $\vect{y}$. This is achieved by using the obtained posterior distribution of the source (population) data as a GP prior for the GP of the ADAS-Cog13 scores of the target subject at time $t$, given by $p(\vect{y}^{\bf(t)}_{t}|\vect{x}^{\bf(t)}_t, \{\vect{X}^{\bf(s)},\vect{Y}^{\bf(s)}\}, \vect{\theta})$. Assuming that we have already observed the data of the target subject, up to time $t-1$, we use the data pairs $\{\vect{x}^{\bf(t)}_{1:t-1},\vect{y}^{\bf(t)}_{1:t-1}\}$ to correct the posterior distribution for the target subject's data at time $t$. More formally, by applying Eqs.~(\ref{post_mu}--\ref{post_s}) to $\{\vect{x}^{\bf(t)}_{1:t-1},\vect{y}^{\bf(t)}_{1:t-1}\}$, we obtain the conditional prior on the target subject data, $\mathcal{N}(\label{prior_mut}\vect{\mu}_{_{1:t-1}}^{\bf(t|s)},\label{prior_{st}}\vect{V}_{1:t-1}^{\bf(t|s)})$, where $\bf(t|s)$ denotes the conditioning order. For exact derivation, see~\citep{liu2015bayesian}. Given this prior and a test input $\vect{x}_\ast=\vect{x}_{t}^{\bf(t)}$, the correct form of the adapted posterior after observing the target subject data at visit $t$ represents the predictive distribution of the personalized GP (pGP):\\
\scalebox{1}{\parbox{1\linewidth}{%
\begin{align}
\small
\label{post_muad}\vect{\mu_{\ast}^{\bf(p)}} &= \vect{\mu_\ast^{\bf(s)}} +  {\vect{V}_{\ast}^{\bf(t|s)}}^T (\vect{V}_{1:t-1}^{\bf(t|s)} + \sigma_{\bf s}^2\vect{I})^{-1}(\vect{Y}^{\bf(t)}_{1:t-1} - \vect{\mu}_{1:t-1}^{\bf(t|s)}) \\
\label{post_sad}\vect{V_{\ast}^{(p)}} &= \vect{V_{\ast}^{(s)}} -
{\vect{V}_{\ast}^{\bf (t|s)}}^T (\vect{V}_{1:t-1}^{\bf(t|s)} + \sigma_{\bf s}^2\vect{I})^{-1}
\vect{V}_{\ast}^{\bf(t|s)},
\end{align}
}}\\
where {\small $\vect{V}_{\ast}^{\bf(t|s)} = k(\vect{x}^{\bf(t)}_{1:t-1}, \vect{x}_\ast) -
{k(\vect{X}^{\bf(s)}, \vect{x}^{\bf(t)}_{1:t-1})}^T (\vect{K}^{\bf(s)} + \sigma_{\bf s}^2\vect{I})^{-1} k(\vect{X}^{\bf(s)}, \vect{x}_\ast)$}. Eqs.~(\ref{post_muad}--\ref{post_sad}) show that final forecast by the pGP is the combination of the population-model forecast, plus a correction term.\footnote{For $t$=$1$, the population model is used instead, and from $t$=${\vect T}-4$:${\vect T}$ the future target scores are repeated to fill in the window of target scores.} The latter shifts the mean toward the distribution of the target subject and improves the model's confidence by reducing its predictive variance as more data of the target subject is observed. The inference in pGP is efficient as it uses the kernel parameters of the sGP, thus, no further training is employed.

We also form the target-subject-specific GP (tGP) model using only the observed data of the target subject up to time $t$, i.e., the same data we used for the adaption in pGP. However, since this data set is of insufficient size to train the GP parameters -- the GP would easily overfit -- we use the kernel parameters of sGP. Unlike sGP, which has a fixed covariance matrix (trained using the source subjects), tGP continually updates its covariance matrix as more past data (up to $t$) of a target subject become available. Nevertheless, the inference procedure is the same, and for $\vect{x}_\ast=\vect{x}_t^{\bf(t)}$, the predictive distribution of tGP is given by:

\scalebox{1}{\parbox{1\linewidth}{%
\begin{align}
\small
\label{post_mut}\vect{\mu}_{\ast}^{\bf (t)} &= {\vect{k}_{*}}^T (\vect{K}_{1:t-1}^{\bf (t)} + \sigma_s^2\vect{I})^{-1}\vect{y}^{\bf (t)}_{1:t-1} \\
\label{post_st}\vect{V}_{\ast}^{\bf (t)} &= k_{**} - {\vect{k}_{*}}^T (\vect{K}_{1:t-1}^{\bf (t)} + \sigma_s^2\vect{I})^{-1}
\vect{k}_{*}.
\end{align}
}}
where {\small $\vect{k}_\ast = k(\vect{x}^{\bf(t)}_{1:t-1}, \vect{x}_\ast)$}, {\small $k_{\ast\ast} = k(\vect{x}_\ast, \vect{x}_\ast)$}, and $\vect{K}_{1:t-1}^{\bf(t)}=k(\vect{x}^{\bf(t)}_{1:t-1}, \vect{x}^{\bf(t)}_{1:t-1})$. Note that the kernel matrix here increases in size after each visit of a target subject.

\subsection{Personalized GP Experts (pGPE)}
\label{pgpe}
After obtaining the pGP and tGP expert models, an optimal weighting scheme is learned to combine these two models within the proposed pGPE approach. We introduce a novel weighting scheme based on meta-learning that assigns the optimal weights to the expert models for each subject and his/her visit. More formally, recall that the predictive distribution of each expert's forecasts for the time window ahead is given by:

\begin{equation}
\text{pGP:} \,\vect{y}^{\bf(p)} \sim {\mathcal{N}}(\vect{\mu^{(p)}},\vect{V^{(p)}}), \,\,\,\,\, \text{and} \,\,\,\text{tGP:}\,\vect{y^{(t)}} \sim {\mathcal{N} }(\vect{\mu^{(t)}},\vect{V^{(t)}}).
\end{equation}
Using the expert distributions, we obtain the forecasts $\vect{y^{(g)}}$ by the pGPE model as:
\begin{equation}
\vect{y^{(g)}} = \alpha \vect{y^{(p)}} + (1 - \alpha )\vect{y^{(t)}} \sim \mathcal{N}(\alpha \vect{\mu^{(p)}} + (1 - \alpha )\vect{\mu^{(t)}},{\vect{\alpha}^2}\vect{V^{(p)}} + {(1 - \alpha )^2}\vect{V^{(t)}})
\end{equation}
In this work, we consider only the point predictions, defined as the mean of the predictive distribution defined above. To find an optimal weight $\alpha$ for each subject and his/her visit, we solve the following optimization problem:

\begin{equation}
\label{sq}
{\vect{\alpha} ^{opt}} = \mathop {\arg \min }\limits_{{\alpha _i}} \frac{1}{4N}\sum\limits_{i = 1}^N \sum\limits_{k = 1}^4 {{{\left\| {{\vect{y}^{\bf(g)}_{i,k}} - {\vect{y}_{i,k}}} \right\|}^2}},
\end{equation}
where $\vect{\alpha}^{opt}=[\alpha^{opt}_1,\dots,\alpha^{opt}_N]$, and $N=N_s\times {\vect T}$ is the total number of training samples from the training subjects and their visits. Since we seek to optimize one $\alpha$ for each data sample, Eq.\ref{sq} is a convex optimization problem with a closed form solution given by:
\begin{equation}
\label{eq:weights}
\alpha _i^{opt} = \sum\limits_{k = 1}^4\frac{\vect{y}^{\bf(g)}_{i,k} - \vect{y}_{i,k}} {\vect{y}^{\bf(p)}_{i,k} - \vect{y}^{\bf(t)}_{i,k}}\,\,\,\, , i=1\dots N.
\end{equation}
These optimal weights are used for further learning in the proposed approach. While we are able to learn these weights for the training data, it is unclear how to generate those weights for the target subjects and their visits since the future scores are unknown. To solve this, we first derive the following feature vector for each visit of the training subjects:
\begin{equation}
\vect{m}_t = [\vect{y}^{\bf(p)}_t\,\vect{y}^{\bf(t)}_t\,y_t], \,\,\dim (\vect{m}_t) = 1 \times 9,
\end{equation}
where $\vect{m}_t$ contains the four predictions per expert, pGP ($\vect{y}^{\bf(p)}$) and tGP ($\vect{y}^{\bf(t)}$), obtained by evaluating the pre-trained expert models on the data of the training subjects' visits $t=1,\dots,{\vect T}$. We also include their ground truth score for the current visit, $\vect{y}_t$. However, we do not use the data of the target subjects. Finally, the expert weights for the new data point at which we aim to forecast the future ADAS-Cog13 are obtained as:
\begin{equation}
{\alpha _*} = f({\vect{m}_*}) + {\epsilon _*},
\end{equation}
where the function $f(\cdot)$ is trained using a GP regression model with the RBF-ard kernel function and training data pairs $\{\vect{M},{\vect{\alpha} ^{opt}}\}$, where $\vect{M}=\{\vect{m}_{i=1,..,N}\}^T$. Note that the feature vector was centered by subtracting the mean of its elements. Also, we applied the quadratic expansion of such feature vector followed by the element-wise normalization with the norm of the expanded feature vector, resulting in a 55-dim feature vectors. This allowed the GP to capture the underlying relationships between these meta-features and optimal weights for the target experts. We also experimented with other feature types (e.g., using the input features, the prediction uncertainty, and the optimal weights from previous visits of the target subject), but the ones reported here achieved the best performance in the task.
\section{Results}
\label{exp}
Data used in this paper come from the ADNI database \href{http://adni.loni.usc.edu/}{(adni.loni.usc.edu)}. We downloaded the standard dataset processed for the TADPOLE Challenge~\citep{tadpole2017}; this dataset represents 1,737 unique subjects and was created from the \href{https://adni.bitbucket.io/index.html}{ADNIMERGE} spreadsheet, to which regional MRI (volumes, cortical thickness, surface area), PET (FDG, AV45, AV1451), DTI (regional means of standard indices) and cerebrospinal fluid (CSF) biomarkers were added. From this data, we excluded: the PET data, because of their sparsity, the cognitive scores other than ADAS-Cog13, as well as the clinical status (CS) and normalized ventricle volumes (ICVn)\footnote{In the TADPOLE Challenge, ADAS-Cog13, CS and ICVn are treated as target outputs, so in this work we excluded the latter two from the predictors in our model.}, to construct a multi-modal feature set containing: demographics (6 features), genetics (3 features), CSF (3 features), MRI (365 features), and DTI (229 features), thus 606 input features in total. Furthermore, we selected a cohort of subjects with at least $10$ visits, and whose records were not missing more than $82.5\%$ of the subject data (from all modalities/visits), resulting in 100 subjects, 48 of whom were diagnosed with AD. We performed a 10-fold subject-independent cross-validation (i.e., each fold contained data of 10 subjects).\footnote{The models were trained using data from 9 folds, and tested on data of the remaining fold. This was repeated for each of 10 folds.} The input features were z-normalized (zero mean, unit variance). To measure the models' performance, we report $\mu\pm\sigma$ of mean absolute error (MAE) for the 10-folds. To form the forecasting window of four consecutive visits (6, 12, 18 and 24 months), we imputed the ADAS-Cog13 scores for the missing visits using the scores of the past visits. However, the reported evaluation metrics were computed using the scores for the existing visits only. We evaluated three types of GP models: (i) population-level (sGP), (ii) personalized (pGP and tGP), and (iii) personalized experts (pGPE). The settings for these models are as follows:
\begin{itemize}
\item {\bf Population-level GPs.} To forecast the future ADAS-Cog13 scores, we trained the sGP models using the input features for the visits up to time $t$ and the corresponding ADAS-Cog13 score ($y_t$). We employed two kernel functions: sGP(RBF-iso) and sGP(RBF-ard)~\citep{rasmussen2006gaussian}, where the latter learns a separate length-scale parameter for each data modality used, and $y_t$, i.e., ${\lambda _{i = 1:6}}$. We also report the performance of the sGP(RBF-iso) model when only $y_t$ from the current visit (VIS) is used as a predictor of the future scores, effectively doing the label smoothing. We denote this model as sGP(VIS).

\begin{table}
\caption{Comparison of different forecasting approaches. All the differences between pGPE and the compared models are statistically significant ($p <0.001$) based on a paired t-test with equal variances. The last row shows a lower bound on MAE, obtained when the ADAS-Cog13 scores from the {\it target} subjects are used to determine the optimal weights (W$_{opt}$) for the expert models (Sec.~\ref{pgpe}). In pGPE, the meta-weights (W$_{reg}$) for target subjects are estimated with GP tuned on {\it training} subjects.}
\label{tab_res}
\centering
\begin{center}
\begin{adjustbox}{width=0.7\textwidth}
\begin{tabular}{l|ccccc}
  \hline
    \rowcolor{Gray}
\textbf{Models}        & &&\textbf{MAE}& & \\
\hline
{}              & \text{$t+1$}& \text{$t+2$} & \text{$t+3$}& \text{$t+4$} & \text{Avg.}
                \\
\hline
\toprule
\textbf{Base ($y_t$)} &3.74$\pm$0.17 &3.84$\pm$0.17 &4.72$\pm$0.29 &5.02$\pm$0.33 &4.34\\
\toprule
\textbf{LassoR} &3.76$\pm$0.18 &3.85$\pm$0.16 &4.61$\pm$0.26 &4.51$\pm$0.27&4.20\\
\textbf{SVR(RBF-iso)} &3.97$\pm$0.41 &4.49$\pm$0.57 &4.88$\pm$0.49 &5.21$\pm$0.66 &4.63\\
\textbf{LSTM} &4.87$\pm$0.31 &5.44$\pm$0.36 &5.52$\pm$0.42 &6.48$\pm$0.53 &5.58\\

\toprule
\textbf{sGP(VIS)} &3.62$\pm$0.33 &3.67$\pm$0.31 &4.37$\pm$0.54 &4.41$\pm$0.54 &4.02\\
\textbf{sGP(RBF-iso)} &3.86$\pm$0.37 &4.39$\pm$0.52 &4.72$\pm$0.44 &5.04$\pm$0.62 &4.50\\
\textbf{sGP(RBF-ard)} &3.50$\pm$0.30 &3.55$\pm$0.38 &4.11$\pm$0.61 &4.27$\pm$0.77 &3.86\\
\toprule
\textbf{pGP(RBF-ard)} &3.50$\pm$0.31 &3.49$\pm$0.35 &4.00$\pm$0.52 &4.04$\pm$0.63 &3.76\\
\textbf{tGP(RBF-ard)} &3.70$\pm$0.37 &4.07$\pm$0.54 &3.90$\pm$0.42 &4.38$\pm$0.79 &4.01\\
\toprule
\textbf{pGPE(W$_{prior}$)} &3.56$\pm$0.34 &3.47$\pm$0.40 &3.97$\pm$0.51 &4.04$\pm$0.65 &3.76\\
\textbf{pGPE(W$_{freq}$)} &3.70$\pm$0.37 &3.33$\pm$0.34 &3.67$\pm$0.40 & 3.82$\pm$0.59 &3.63\\
\textbf{pGPE(W$_{ave}$)} &3.47$\pm$0.31 &3.46$\pm$0.38 &3.64$\pm$0.41 &3.85$\pm$0.60 &3.61\\
\textbf{pGPE(W$_{var}$)} &3.66$\pm$0.36 &3.98$\pm$0.51 &3.85$\pm$0.41 &4.29$\pm$0.75 &3.95\\
\textbf{pGPE(W$_{reg}$)} &{\bf 2.66$\pm$0.28} &{\bf 2.57$\pm$0.24} &{\bf 2.48$\pm$0.29} &{\bf 2.87$\pm$0.32}&{\bf 2.65}\\
\toprule
\rowcolor{Gray}
\textbf{pGPE(W$_{opt}$)} &1.80$\pm$0.28 &1.50$\pm$0.14 &1.49$\pm$0.17 &1.54$\pm$0.37 &1.58\\
\bottomrule
\end{tabular}
 \end{adjustbox}
\end{center}
\end{table}
\item {\bf Personalized GPs.} We also report the performance by the pGP and tGP models (Sec.~\ref{methods}). The former performs the adaptation of the sGP to the new subject using the subject data up to time $t$, while the latter builds a new kernel matrix using only the data of the new subject (however, it uses the hyper-parameters of the sGP(RBF-ard) model). We used the sGP(RBF-ard) model described above to derive the pGP and tGP models, since it achieved the best performance among the population-level GPs.
\item {\bf Personalized GP Experts.} As described in Sec.~\ref{pgpe}, we used the pGP/tGP models as experts in the proposed pGPE approach. Below we describe different weighting schemes, including the proposed meta-weighting, we applied to these experts.\\
{\bf W$_{prior}$}: derived by inspecting the {\it average} performance of the pGP/tGP models and by applying the best performing model per visit/forecasting step (see Fig.\ref{prior}).\\
{\bf W$_{freq}$}: derived based on the frequency of pGP outperforming tGP per visit and per subject for each forecasting step. The weights range from 0 to 1, and represent the normalized sum over the columns (subjects) depicted in Fig.\ref{freq}.\\
{\bf W$_{ave}$}: derived by averaging the forecasts by pGP and tGP, as in~\citep{utsumi2018pgp}.\\
{\bf W$_{var}$}: the variance-based weighting~\citep{deisenroth2015distributed} of pGP/tGP.\\
{\bf W$_{reg}$}: the proposed regression-based meta-weighting of pGP/tGP (Sec.~\ref{pgpe})\\
{\bf W$_{opt}$}: the optimal weights obtained by applying Eq.~(\ref{eq:weights}) to the target data.
\end{itemize}
We also include the baseline comparisons with non-GP models, commonly used for forecasting tasks, and usually applied to the ADNI dataset. Specifically, we compare to Lasso Regression (LassoR), Support Vector Regression (SVR) with RBF(iso), and Long Short-Term Memory (LSTM)~\citep{hochreiter1997long}. In relation to the GP models evaluated, LassoR can be seen as our sGP with linear kernel and $l_1$ regularization, and SVR as a sparse version of our sGP. However, SVR does not provide probabilistic outputs, and, thus, it lacks a principled way for model adaptation, as done in our personalized GP models via their posterior distribution. The optimal regularization parameter and kernel width of LassoR and SVR, respectively, were selected on the held-out subjects from the training set (10/90). As input to LSTM, we tried a sliding window of the size $n=1,..,5$, containing observations from the past visits. We used again the held-out dataset to select the optimal window size ($n=2$), and the number of the LSTM output states ($h=64$). This was followed by a fully-connected linear layer ($64\times4$). During the model training, we applied a dropout (0.3) to prevent the model overfitting. Lastly, we report the results obtained by a model-free approach, denoted as Base ($y_t$), where the score from the current visit ($y_t$) was used as a forecast of the four steps ahead.

Table 1 compares the evaluation metrics for the model settings described above. By comparing different population-level GP models, we note that simple smoothing of the previous ADAS-Cog13 score ($y_t$) using sGP(VIS) is quite effective. We attribute this to the fact that the target scores from the consequent visits are highly correlated. Compared to the model free approach, Base ($y_t$), which forecasts by "carrying forward" the current score, we note that sGP(VIS) takes advantage of the GP's smoothing property, effectively leveraging the labels (i.e. ADAS-Cog13 scores) from the training population. This results in average error reduction of 0.32 over the four time steps.
However, the data-informed sGP(RBF-ard) model (that performs the smoothing of the input features) achieved the best performance overall. By learning the separate length-scale parameters for each modality, this model was able to successfully combine different data types - something that cannot be achieved with RBF-iso, resulting in the adverse performance by the sGP(RBF-iso) model. Next, we compare the personalized models (pGP and tGP) that use the sGP(RBF-ard) as the base model. It is evident that using the domain adaptive GPs to personalize the population-level model to each subject (pGP) helps to reduce MAE, the difference being more pronounced in later time steps. On the other hand, the tGP model outperforms the population-level sGP/pGP only at step $t+3$. Yet, as can be observed from Fig.~\ref{prior}, tGP outperforms these models at certain visits, even though this is not reflected in its MAE.

The bottom half of Table 1 shows results obtained from different weighting schemes that we devised to compare with our regression-based pGPE approach (pGPE(W$_{reg}$)). First, note that the weighting of the pGP/tGP models based on the heuristics derived by looking at the models' scores per-visit/subject leads to similar or better performance compared to the pGP model alone. Specifically, frequency-based weighting (W$_{freq}$) is slightly more effective than W$_{prior}$ as it is not biased by the amplitude of the per-subject MAEs when deriving the weights. This is because some subjects had much higher MAE than the others, which, evidently, did not translate well to test subjects when applying this binary model selection encoded by W$_{prior}$. Interestingly, averaging the two experts (pGP/tGP) via W$_{ave}$ further reduces MAE. As this model outperforms both the pGP and tGP alone, this also signals that the two models have very different performance per subject/visit, also evidenced by Figs.~\ref{prior}\&\ref{freq}. Thus, by simply averaging their predictions, this model reduces the prediction biases of the pGP/tGP models. We also explored the predictive variance of the pGP/tGP experts to assign different weights to the models based on their confidence in the target predictions~\citep{deisenroth2015distributed}. While this approach improved the performance of tGP alone, it hurt the overall performance of pGP. By looking into the estimated variances, we found that tGP tended to overestimate the prediction variance, due to the small amount of data (only the target subject visits' data were used to construct the kernel).

\begin{figure}[!tbp]
  \centering
{\includegraphics[width=0.35\textwidth]{adas13_1.png}\label{fig:f1}}

{\includegraphics[width=0.35\textwidth]{adas13_2.png}\label{fig:f2}}

{\includegraphics[width=0.35\textwidth]{adas13_3.png}\label{fig:f3}}

{\includegraphics[width=0.35\textwidth]{adas13_4.png}\label{fig:f4}}
  \caption{Average MAE for the pGP and tGP models computed per visit, showing specific time-steps where one of the expert models dominates over the other. This is used to derive prior weights (W$_{prior}$) for the pGPE approach by selecting the optimal model (on average) for each visit and across the four forecasting time steps. Note that after the fifth visit, the subjects in ADNI were seen on a yearly basis.}
  \label{prior}
\end{figure}

\begin{figure}[!tbp]
  \centering
{\includegraphics[width=0.2\textwidth]{em_adas13_ts_1.png}\label{fig:f1_b}}

{\includegraphics[width=0.2\textwidth]{em_adas13_ts_2.png}\label{fig:f2_b}}

{\includegraphics[width=0.2\textwidth]{em_adas13_ts_3.png}\label{fig:f3_b}}

{\includegraphics[width=0.2\textwidth]{em_adas13_ts_4.png}\label{fig:f4_b}}
  \caption{Plots depicting the best performing personalized model for each subject and his/her visit. For pGP, this is shown in white (-1), and for tGP this is shown in black (+1). When there is no visit data available, this is shown in gray (0). Note that the pGP always outperforms tGP at $t+1$, while for the later time steps, the models exhibit largely heterogeneous performance.}
  \label{freq}
\end{figure}

\begin{figure}[t!]
  \centering
{\includegraphics[width=0.35\textwidth]{AvgAdasCog13_withVar_CN.png}\label{fig:f1_c}}
{\includegraphics[width=0.35\textwidth]{AvgAdasCog13_withVar_CN_MCI.png}\label{fig:f2_c}} \\
{\includegraphics[width=0.35\textwidth]{AvgAdasCog13_withVar_MCI.png}\label{fig:f3_c}}
{\includegraphics[width=0.35\textwidth]{AvgAdasCog13_withVar_AD.png}\label{fig:f4_c}}
  \caption{The graphs showing the ADAS-Cog13 scores for each visit (averaged across subjects) by the proposed regression-based pGP expert model (pGPE(W$_{reg}$)) -predicted- and ground-truth scores. Each graph shows these scores along with their standard deviation across the subjects, where the subjects were separated based on their clinical status: cognitively normal (CN), those who converted at one point from CN to mild-cognitive impairment (MCI) (CN$\rightarrow$MCI), those who stayed at MCI, and those who were diagnosed initially or converted to AD.}
  \label{convert}
\end{figure}
The proposed regression-based weighting outperforms the compared weighting schemes by large margin, as can be seen from the per-time-step and overall results. This is consistent for the whole forecasting horizon. Note that, apart from the variance-based weighting, the other weighting approaches apply the same and fixed expert weights across all subject. This is suboptimal. The benefit of learning the subject- and visit-specific weights, as introduced here, is clear from the results obtained. This further shows that the optimal weights vary largely across subjects/visits (see Fig.~\ref{freq}); however, the proposed meta-weighting using GP was able to effectively learn those changes. To confirm this interpretation, we applied random weights sampled from the learned GP. This resulted in performance significantly lower than that obtained by the population-based models. We also included the results obtained with the optimal weights (W$_{opt}$ -- see Sec.~\ref{pgpe}) that show the lower-bound on MAE that can be achieved with the proposed pGPE approach. By looking at the performance achieved by the baselines, we note that LassoR, typically used in clinical settings because of its interpretability, outperforms the GP and SVR population models with non-linear mappings (RBF-iso). This evidences that a simple linear model can be useful in this context, however, like SVR that is commonly used in prediction tasks in ADNI, it does not provide a probabilistic output. On the other hand, due to the probabilistic nature of GPs, pGP can easily adapt to the target subjects, which result in lower average MAE. While LSTMs have shown great performance in various clinical time series forecasting tasks, in our experiments on the ADNI data, it underperformed. We attribute this to the highly heterogenous and irregularly sampled data, which, evidently, negatively affects the ability of LSTM to leverage its memory properties.

Fig.~\ref{convert} shows the graphs depicting the predicted ADAS-Cog13 scores for each visit (averaged across subjects) by the proposed pGPE(W$_{reg}$) that we plot against the ground-truth scores. Each graph shows these scores along with their one standard deviation across the subjects, where the subjects were separated based on their clinical status (not used as input in the models): 25 cognitively normal (CN), 20 who converted at one point from CN to mild-cognitive impairment (MCI) (CN$\rightarrow$MCI), 7 who stayed at MCI, and 48 who were diagnosed initially or converted to Alzheimer's Disease (AD) in one of their visits. Note that the proposed approach can accurately forecast the trend in changes of the target scores. As can be seen, the subjects who were diagnosed with AD or converted to AD at one of the visits have a large deviation from the ADAS-Cog13 scores for the other three types of clinical status, with this difference being more pronounced towards later visits, as expected. While we show here that the proposed approach matches well the distributions of the cognitive scores at the sub-group level, the proposed approach can be used to predict the trend of the cognitive changes for each target subject, and associate it with future changes in his/her clinical status (e.g. to detect subjects that will rapidly progress over a period of 2 years). We plan to investigate the feasibility of such approach in our future work.

\section{Conclusions}
\label{headings}
Accurate forecasting of changes in the biomarkers of AD is challenging, mainly because of the highly heterogeneous, noisy and missing nature of the clinical data available for this task, but also due to the highly pronounced individual differences in such data. As a step forward, this work developed a novel personalized approach for accurate forecasting of ADAS-Cog13, a significant biomarker of AD, up to two years in the future. The proposed personalized GP framework for time series forecasting uses the notion of meta learning to determine an optimal weighting scheme for individual expert models based on personalized GP models. We showed on a sub-cohort of ADNI subjects that this approach brings significant improvements over population-level GP models. Furthermore, we showed that the proposed personalized experts alone cannot generalize well enough across all the subjects and their visits. By contrast, the introduced personalized meta-weighted GP framework enables accurate forecasting of changes in ADAS-Cog13, outperforming the existing personalized GP models, and traditional models commonly used for forecasting of time-series clinical data. We also showed that the newly introduced meta-weighting is more effective than the standard weighting schemes for GP expert models. In the future, we plan to extend our meta-weighted GPE framework so that it can handle more than two experts and forecast more than one biomarker of AD simultaneously. For this, we will investigate the notion of multi-task GPs and reinforcement learning, for actively learning to select the optimal experts for each subject/visit. We also plan to employ techniques for filling in the missing data in order to increase the cohort of the subjects from ADNI for evaluation of our models. This automated approach for forecasting of cognitive changes in AD could augment and assist clinicians by providing them with intelligent data summarization and decision support tools for early identification of at-risk subjects and construction of informative clinical trials.

\small{