Stability and Bounded Real Lemmas of Discrete-Time MJLSs with the Markov Chain on a Borel Space
Chunjie Xiao Ting Hou Weihai Zhang




                                              
                                              
                                              

 
                                                

[footnoteinfo]Corresponding author: Ting Hou. Tel. +86-13455248030.

sdnu]Chunjie Xiaoxiaocj6_sd@163.com,
sdnu]Ting Houht_math@sina.com,
sdust]Weihai Zhangw_hzhang@163.com,


[sdnu]College of Mathematics and Statistics, Shandong Normal University,
Jinan 250014, Shandong Province, China
[sdust]College of Electrical Engineering and Automation, Shandong University of Science and Technology,
Qingdao 266590, Shandong Province, China



                           
Markov jump linear systems; Borel space; exponential stability;  bounded real lemma.                       
                             
                                          
                                          


             
In this paper, exponential stability of discrete-time Markov jump linear systems (MJLSs)
with the Markov chain on a Borel space (Θ, ℬ(Θ))
is studied, and bounded real lemmas (BRLs) are given.
The work generalizes the results from the previous literature that considered only the Markov chain taking values in a countable set
to the scenario of an uncountable set and provides unified approaches for describing exponential stability and H_∞ performance of MJLSs.
This paper covers two kinds of exponential stabilities: one is exponential mean-square stability with conditioning (EMSSy-C),
and the other is exponential mean-square stability (EMSSy).
First, based on the infinite-dimensional operator theory,
the equivalent conditions for determining these two kinds of stabilities are shown respectively by the exponentially stable evolutions
generated by the corresponding bounded linear operators on different Banach spaces,
which turn out to present the spectral criteria of EMSSy-C and EMSSy.
Furthermore, the relationship between these two kinds of stabilities is discussed. Moreover,
some easier-to-check criteria are established for EMSSy-C of MJLSs in terms of the existence of
uniformly positive definite solutions of Lyapunov-type equations or inequalities.
In addition, BRLs are given separately in terms of the existence of solutions of
the Θ-coupled difference Riccati equation for the finite horizon case
and algebraic Riccati equation for the infinite horizon case,
which facilitates the H_∞ analysis of MJLSs with the Markov chain on a Borel space.






§ INTRODUCTION


Switching systems are usually considered to model real-world systems that undergo sudden changes in structures
due to environmental disturbances and component failures or repairs.
As a special kind of stochastic switching systems governed by Markov processes,
Markov jump linear systems (MJLSs) have gained much attention,
and some theoretical results on stability, observability, detectability, state estimation, robust control, and optimal control
have been developed for this kind of systems.
We do not intend to be exhaustive here but only mention
<cit.>
as well as the references therein as several samples of the theoretical studies on issues related to this paper.
Moreover, applications of the above theoretical results can be found in a wide range of fields,
including communications (see, <cit.>), robotics (see, <cit.>),
and new energy generation (see, <cit.>).

In real-world systems, the presence of exogenous disturbance often leads to oscillation or even results in worsen control quality.
To investigate the disturbance attenuation for disturbed systems, in recent decades,
H_∞ analysis has been the topic of extensive investigation.
As a key result in H_∞ analysis,
the infinite horizon bounded real lemma (BRL) in time-domain analysis estimates the H_∞ norm
of perturbation operator for the internally stable systems
relying on the existence of stabilizing solutions of the algebraic Riccati equations (AREs) or the viability of some linear matrix inequalities.
Further, it is important that BRLs have been widely applied in filter and controller designs.
For instance, BRLs were presented in <cit.> for homogeneous and in <cit.> for nonhomogeneous MJLSs
where the Markov chain takes values in a finite set (finite MJLSs).
Regarding the applications in state estimation, the H_∞ filter of MJLSs with the piecewise homogeneous Markov chain
was designed in <cit.> by presenting the BRL for the corresponding filtering error systems.
As for H_∞ control problems, after providing a BRL for H_∞ performance analysis of MJLSs with partial observations,
a detector-based H_∞ controller design method was introduced in <cit.>.

The literature mentioned above focused only on analysis and synthesis problems of finite MJLSs,
and in effect, most of the existing literature on MJLSs did as well.
However, one should note that different Markov chain state spaces might result in different research findings.
For example, the notions of stochastic stability and mean-square stability are equivalent for finite MJLSs (see, <cit.>),
but no longer equivalent when the Markov chain state space is countably infinite.
One may refer to <cit.> and <cit.> for the cases of continuous-time and discrete-time systems, respectively.
For that reason, the control problems of MJLSs with countably infinite Markov jump parameters have been studied intensively
(see, <cit.>).
Regardless of whether the state space of a Markov chain is a finite set or a countably infinite set,
it just takes values in a discrete set, and none of the above results can be directly applied to MJLSs with the continuous state Markov chain
due to the difference between the sum of finite or countably infinite terms and the integral.
However, in real-world applications, a continuous state Markov chain might be more suitable for describing some general phenomena,
such as the continuous-valued random delays that exist in networked control systems
randomly determined by the last few delays (see, <cit.>).
Naturally, when the Markov chain is on a Borel space,
which is a general (not necessarily countable, e.g., uncountable) state space, including the case of continuous state space,
some general findings on analysis and synthesis of MJLSs have been presented over the last decade.
For example, <cit.> focused on mean-square stability,
uniformly exponential almost sure stability, and stochastic stability of MJLSs, respectively.
In addition, some meaningful problems of MJLSs,
such as LQ control problems (see, <cit.>) and filtering problems (see, <cit.>),
were studied.

In this paper, we pay attention to exponential stability and disturbance attenuation property of discrete-time MJLSs
with the Markov chain on a Borel space (Θ, ℬ(Θ)).
The main contributions are given as follows:


  * This paper extends the previous results of MJLSs with the Markov chain on a countable set to an uncountable set.
      The extension is achieved based on the analysis of the Borel-measurable matrix-valued functions with the help of measure theory,
      such as ensuring the integrability of solutions of Θ-coupled Lyapunov-type equations and AREs,
      which makes an essential difference from the previous literature.
      In fact, the Markov chain state space could be finite or countably infinite,
      only that these two special cases are no longer the key focus of our study.
      For one thing, as mentioned earlier, these two cases have been intensively studied.
      For another thing, under the analytical framework established in this paper,
      some results obtained previously in the literature for these two cases are readily apparent.

  * Based on the infinite-dimensional operator theory, this paper establishes the equivalence
      between the exponential mean-square stability with conditioning (EMSSy-C) of MJLSs and the  exponentially stable anticausal evolution (ESAE)
      generated by the bounded linear operator 𝒯_A on Banach space 𝕊ℍ_∞^n.
      In fact, it is reasonable to propose the concept of EMSSy-C because in some real-world systems
      which subsystem the switching system belongs to at the initial moment is directly available or even achievable by control.
      Moreover, a spectral criterion and some easier-to-check Lyapunov-type criteria are established for testing EMSSy-C of MJLSs. Regarding exponential mean-square stability (EMSSy), a criterion is given by the exponentially stable causal evolution (ESCE) generated by
      operator ℒ_A on Banach space 𝕊ℍ_1^n.
      As a result, a spectral criterion for EMSSy of MJLSs is directly obtained.

  * The BRLs are characterized by the Θ-coupled difference Riccati equation (DRE) for the finite horizon case
      and ARE for the infinite horizon case (regarding the Θ-coupled ARE, one can refer to <cit.>),
      which develops a general method for the H_∞ analysis of MJLSs with the Markov chain
      on a Borel space (Θ, ℬ(Θ)) in a unified way.
      For the infinite horizon case,
      it is proved that internal stability of the system guarantees external stability,
      and therefore the associated perturbation operators can be given.
      To measure the effect of unknown perturbations on the output in the worst-case scenario for MJLSs,
      the H_∞ norm of the given perturbation operator is no longer provided by a sup operator
      but is given by an  operator which appears to be new.
      Furthermore, to the best of the authors' knowledge,
      this is the first time that a sufficient and necessary condition is presented for a given MJLS
      with the Markov chain on a general Borel space
      to ensure that the H_∞ norm of the associated perturbation operator is below a prescribed level γ>0.
      Moreover, we set up an iterative algorithm for solving the associated Θ-coupled ARE.
      Besides, it is noteworthy here that H_∞ analysis is the basis of H_∞ synthesis.
      The BRLs obtained can be subsequently applied to solve the system synthesis problems
      such as H_∞ filter and H_∞ controller designs of MJLSs.


The remainder is arranged as follows:
In Section <ref>,
the dynamic model of MJLSs with the Markov chain on a general Borel space (Θ, ℬ(Θ))
and some auxiliary results on the bounded linear operators are described;
Section <ref> deals with exponential stability;
In Section <ref>, we further explore H_∞ performance of the systems,
and BRLs in the finite and infinite horizon are derived;
Examples are given in Section <ref>;
Section <ref> finishes the paper with some concluding remarks.



§ PRELIMINARIES AND AUXILIARY RESULTS


Notations.
ℕ^+:={1,2,⋯};
ℕ:={0}⋃ℕ^+;
for integers i and j, i,j:={i,i+1,i+2,⋯,j};
ℝ^n is the n-dimensional real Euclidean space with the Euclidean norm ·.
For Banach spaces 𝕄 and 𝕄̅, 𝔹(𝕄,𝕄̅) represents the Banach space of
all bounded linear operators of 𝕄 into 𝕄̅ with the uniform induced norm denoted by ·,
i.e., for ℒ∈𝔹(𝕄,𝕄̅), ℒ:=sup_x∈𝕄, x≠0{ℒx/x};
for simplicity, 𝔹(𝕄):=𝔹(𝕄,𝕄).
Let σ(ℒ) be the spectral set of ℒ∈𝔹(𝕄),
and the spectral radius of ℒ is defined to be r_σ(ℒ):=sup_λ∈σ(ℒ)|λ|. For Banach space 𝕄, a convex cone 𝕄^+⊂𝕄 induces an ordering “≤” on 𝕄
by V≤V̅ if and only if (iff) V̅-V∈𝕄^+.
An operator ℒ∈𝔹(𝕄) is called a positive operator,
denoted by ℒ≥0, if ℒ𝕄^+⊂𝕄^+.
In particular, ℝ^m× n:=𝔹(ℝ^n, ℝ^m) denotes the m × n real matrix space.
As usual, the n× n identity matrix is expressed by I_n× n.
For Q∈ℝ^m× n, Q' means the transpose of Q;
Q:=[λ_max(Q'Q)]^1/2; 𝕊^n:={Q∈ℝ^n× n|Q'=Q}.
For Q_1∈𝕊^n, Q_2∈𝕊^n,
Q_1<Q_2 (Q_1≤ Q_2) indicates that Q_1-Q_2 is a negative definite (negative semidefinite) matrix.
𝕊^n+*:={Q∈𝕊^n|Q≥α I_n× n for some α>0}.
In what follows, we let n be the associated dimension of the matrix, and n is a constant.

The following result will be mentioned repeatedly:
<cit.>
Given any Q∈𝕊^n with Q≥0, we have Q≤ tr(Q) ≤ n Q, where tr(·) denotes the trace operator.


In this paper, Θ is assumed to be a Borel subset of a Polish space (i.e., a separable and complete metric space).
The Borel space (Θ, ℬ(Θ)) is defined as Θ,
together with its Borel σ-algebra ℬ(Θ).
μ is a σ-finite measure on ℬ(Θ).
By Theorem 13.1 in <cit.>, the topology of the Polish space can be extended to
a new topology with the same Borel sets in which Θ is clopen (i.e., open and closed), and so is Polish (in the relative topology).
Further, by Theorem 8.3.6 in <cit.>, if Θ is uncountable (which is the scenario of our interest),
then it is Borel isomorphic to ℝ^1.



On a probability space (Ω, 𝔉, 𝒫),
define a Markov chain {ϑ(k)}_k∈ℕ taking values in Θ
with the initial distribution given by a probability measure μ̂
and the stochastic kernel 𝒢(·|·) satisfying
𝒢(Δ|ϑ(k)):=𝒫(ϑ(k+1)∈Δ|ϑ(k))
almost surely (a.s.),  ∀ k∈ℕ and Δ∈ℬ(Θ).
Suppose that for any ℓ∈Θ,
𝒢(·|ℓ) has a density g(·|ℓ) with respect to μ,
that is, 𝒢(Δ|ℓ)=∫_Δg(s|ℓ)μ(ds) for any Δ∈ℬ(Θ).
We now explain the reasonableness of the assumption on the Markov chain when Θ is Borel.
Indeed, by Theorem 6.3 in <cit.>, the stochastic kernel 𝒢(·|·) exists.
Moreover, the Kolmogorov consistency theorem (for example, see Theorem 10.6.2 in <cit.>)
allows us to construct a Θ-valued Markov chain
with the initial distribution μ̂ and the stochastic kernel 𝒢(·|·).

We say that P={P(ℓ)}_ℓ∈Θ∈ℍ^m× n (P∈𝕊ℍ^n)
if  P(·): Θ→ℝ^m× n (P(·): Θ→𝕊^n) is measurable.
For P, R∈𝕊ℍ^n, P≤ R means that P(ℓ)≤ R(ℓ) holds for μ-almost all ℓ∈Θ.
ℍ^m× n_1={P∈ℍ^m× n| P_1:=∫_ΘP(ℓ)μ(dℓ)<∞};
ℍ^m× n_∞={P∈ℍ^m× n| P_∞:={P(ℓ); ℓ∈Θ}<∞}.
For convenience, ℍ^n_1:=ℍ^n× n_1;
ℍ^n_∞:=ℍ^n× n_∞.
𝕊ℍ^ n_1={P∈𝕊ℍ^n| P_1<∞};
𝕊ℍ^n_∞={P∈𝕊ℍ^n| P_∞<∞}.
In fact, (ℍ_1^m× n,·_1) and (ℍ^m× n_∞,·_∞)
are Banach spaces (see <cit.>).
On Banach space (𝕊ℍ_1^n,·_1),
we consider the order relation induced by the convex cone ℍ_1^n+⊂𝕊ℍ_1^n,
where ℍ_1^n+={P∈𝕊ℍ_1^n|P(ℓ)≥ 0 holds for μ-almost all ℓ∈Θ}.
On Banach space (𝕊ℍ_∞^n,·_∞),
we consider the order relation induced by the convex cone ℍ_∞^n+⊂𝕊ℍ_∞^n,
where ℍ_∞^n+={P∈𝕊ℍ_∞^n |P(ℓ)≥0 holds for μ-almost all ℓ∈Θ}.
It is obvious that (𝕊ℍ_1^n,·_1) and (𝕊ℍ_∞^n,·_∞)
are infinite-dimensional real ordered Banach spaces.
ℍ_∞^n+*={P∈ℍ_∞^n+ |P(ℓ) ≥α I_n× n
holds for μ-almost all ℓ∈Θ, for some α>0}.
Let ℐ={ℐ(ℓ)}_ℓ∈Θ, where ℐ(ℓ)=I_n× n.
ℐ∈ℍ_∞^n+ and  is the identity element of 𝕊ℍ_∞^n.
ℐ_𝕊ℍ_∞^n denotes the identity operator on 𝕊ℍ_∞^n.
Throughout this paper, when analyzing the functions on the given measurable spaces,
all the properties (including all the equations and inequalities) are
in the sense of μ-almost everywhere on Θ (μ-a.e.),
or for μ-almost all ℓ∈Θ, if there is no special reminder.

For any P∈𝕊ℍ_1^n (P∈𝕊ℍ_∞^n),
there exist P^+, P^-∈ℍ^n+_1 (P^+, P^-∈ℍ^n+_∞) such that P=P^+-P^-
and P_1≥max{P^+_1, P^-_1} (P_∞≥max{P^+_∞, P^-_∞}).


For any P∈𝕊ℍ_1^n (P∈𝕊ℍ_∞^n), define
P̂={P̂(ℓ)}_ℓ∈Θ with P̂(ℓ)=P(ℓ) I_n× n.
Clearly, P̂(·) is measurable.
Let
P^+(ℓ)=[P̂(ℓ)+P(ℓ)]/2 and
P^-(ℓ)=[P̂(ℓ)-P(ℓ)]/2, ∀ℓ∈Θ.
The reader can verify that P=P^++P^- may serve as a valid decomposition.


Consider the discrete-time MJLS

    Φ_v:{[      x(k+1)=A(ϑ(k))x(k)+B(ϑ(k))v(k),; y(k)=C(ϑ(k))x(k)+D(ϑ(k))v(k), ∀ k∈ℕ,;              x(0)=x_0∈ℝ^n, ϑ(0)=ϑ_0, ].

where x(k)∈ℝ^n is the system state;
v(k)∈ℝ^r and y(k)∈ℝ^m are the input and output;
x_0 is a deterministic vector in ℝ^n.
𝔉_k denotes the σ-field generated by {x(0), ϑ(0), x(1), ϑ(1), ⋯, x(k), ϑ(k)}.
The following assumptions are given with regard to Φ_v:

(i) A∈ℍ^n_∞, B∈ℍ^n× r_∞,
C∈ℍ^m× n_∞, and D∈ℍ^m× r_∞;

(ii) ϑ(k) is directly accessible at every k∈ℕ;

(iii) C(ℓ)'D(ℓ)=0 holds for μ-almost all ℓ∈Θ;

(iv) The initial distribution μ̂ is absolutely continuous with respect to σ-finite measure μ.


Considering (iv) of Assumption <ref>,
and applying the Radon-Nikodym theorem (for example, see Theorem 32.2 in <cit.>),
it follows that there is ν(ℓ)≥0 satisfying μ̂(M)=∫_Mν(ℓ)μ(dℓ) for any M∈ℬ(Θ),
i.e., ν plays the role of the Radon-Nikodym derivative of μ̂ with respect to μ.
Therefore, we can properly make the assumptions as follows:

 ν(ℓ)>0 μ-a.e..


∫_Θg(ℓ|s)μ(ds)>0 μ-a.e..


In the remainder, we will show several auxiliary results for some given bounded linear operators.
For K∈ℍ^n_∞, V∈ℍ^n_1, U∈ℍ^n_∞, and ℓ∈Θ, define

    ℒ_K(V)(ℓ):=∫_Θg(ℓ|s)K(s)V(s)K(s)'μ(ds),
                ℰ(U)(ℓ):=∫_Θg(s|ℓ)U(s)μ(ds),
                𝒯_K(U)(ℓ):=K(ℓ)'ℰ(U)(ℓ)K(ℓ),

and the bounded bilinear operator by ⟨ V; U⟩:=∫_Θtr(V(s)'U(s))μ (ds).
The following proposition shows some essential properties about the above operators:
<cit.>
For any K∈ℍ^n_∞, V∈ℍ_1^n, and U∈ℍ_∞^n, 

(i) ℒ_K∈𝔹(ℍ_1^n) and r_σ(ℒ_K)≤ℒ_K≤K^2_∞;

(ii) 𝒯_K∈𝔹(ℍ_∞^n) and r_σ(𝒯_K)≤𝒯_K≤K^2_∞;

(iii) ⟨ℒ_K(V); U⟩= ⟨ V; 𝒯_K(U)⟩.


Specially, if we let K∈ℍ^n_∞, V∈𝕊ℍ^n_1, and U∈𝕊ℍ^n_∞,
then ℒ_K and 𝒯_K satisfy ℒ_K(V)∈𝕊ℍ_1^n
and 𝒯_K(U)∈𝕊ℍ_∞^n, respectively.
Indeed, 𝕊ℍ_1^n is an invariant linear subspace of Banach space ℍ_1^n under the operator ℒ_K,
while 𝕊ℍ^n_∞ is an invariant linear subspace of Banach space ℍ^n_∞ under the operator 𝒯_K.
Consequently, we can draw the same conclusions on 𝕊ℍ^n_1 and 𝕊ℍ^n_∞ as Proposition <ref>.
Moreover, ℒ_K and 𝒯_K are positive operators
on ordered Banach spaces 𝕊ℍ^n_1 and 𝕊ℍ_∞^n, respectively.


In what follows, if not specified, ℒ_A and 𝒯_A will be restricted on
𝔹(𝕊ℍ_1^n) and 𝔹(𝕊ℍ_∞^n), respectively.
Here, ℒ_A and 𝒯_A are defined by (<ref>) and (<ref>) with K=A.

For the sake of concentrating on internal stability of Φ_v, let (𝐀;𝒢) represent the following system:

    x̅(k+1)=A(ϑ(k))x̅(k),  ∀ k∈ℕ, x̅(0)=x_0, ϑ(0)=ϑ_0.

Define Γ(k_2,k_1)=A(ϑ(k_2-1))A(ϑ(k_2-2))⋯ A(ϑ(k_1)) for k_1<k_2,
and Γ(k_1,k_1)=I_n× n, as the fundamental matrix solution of (𝐀;𝒢), k_1,k_2∈ℕ.
Thus, ∀ k∈ℕ, x̅(k)=Γ(k,0)x_0.
Define X̅(k)={X̅(k,ℓ)}_ℓ∈Θ, k∈ℕ by

    X̅(k,ℓ)=𝔼{x̅(k)x̅(k)'
    g(ℓ|ϑ(k-1))}, ∀ k∈ℕ^+,

and X̅(0,ℓ)=X_0(ℓ), ∀ℓ∈Θ, where X_0(ℓ)=x_0x_0'ν(ℓ).
Clearly, ∀ k∈ℕ, X̅(k)∈ℍ_1^n+.
Using ℒ_A, the recursive form of X̅(·) is established in the next proposition.
For more details on (i),  please refer to <cit.>, and (ii) follows directly from (i).

(i) For every k∈ℕ, X̅(k)∈ℍ_1^n+ given by (<ref>) satisfies the linear difference equation:
X̅(k+1)=ℒ_A(X̅(k)) with X̅(0)=X_0={X_0(ℓ)}_ℓ∈Θ;

(ii) For every k∈ℕ, X̅(k)=ℒ_A^k(X_0).


The following definitions are from <cit.>.
For all integers k_1≤ k_2, the causal evolution operator S_A(k_2,k_1) generated by ℒ_A is given by S_A(k_2,k_1)=ℒ_A^k_2-k_1 for k_1<k_2
and S_A(k_1,k_1)=ℐ_𝕊ℍ_1^n,
where ℐ_𝕊ℍ_1^n is the identity operator on 𝕊ℍ_1^n.
The anticausal evolution operator T_A(k_1,k_2) generated by 𝒯_A
is given by T_A(k_1,k_2)=𝒯_A^k_2-k_1 for k_1<k_2 and T_A(k_2,k_2)=ℐ_𝕊ℍ_∞^n.

(i) We call that the causal evolution operator S_A(k_2,k_1) generated by ℒ_A is exponentially stable (or ℒ_A generates an ESCE)
if there exist β≥ 1, α∈(0,1) such that
S_A(k_2,k_1)≤βα^k_2-k_1
for all integers k_1≤ k_2;

(ii) We call that the anticausal evolution operator T_A(k_1,k_2) generated by 𝒯_A is exponentially stable (or  𝒯_A generates an ESAE)
if there exist β≥ 1, α∈(0,1) such that
T_A(k_1,k_2)≤βα^k_2-k_1
for all integers k_1≤ k_2.




§ EXPONENTIAL STABILITY


In this section, we will deal with
 exponential stability of (𝐀;𝒢).

(i) We call that (𝐀;𝒢) is exponentially mean-square stable with conditioning (EMSS-C),
if there exist β≥ 1, α∈(0,1) such that
𝔼{.x̅(k)^2|ϑ_0=ℓ}≤βα^kx_0^2 μ-a.e.
for every k ∈ℕ and x_0∈ℝ^n;

(ii) We call that (𝐀;𝒢) is exponentially mean-square stable (EMSS),
if there exist β≥ 1, α∈(0,1) such that
𝔼{.x̅(k)^2}≤βα^kx_0^2
for every k ∈ℕ, x_0∈ℝ^n, and ϑ_0.


According to Definition <ref>, we could unequivocally draw the following result, which shows that EMSSy-C is stronger than EMSSy.

(𝐀;𝒢) is EMSS provided (𝐀;𝒢) is EMSS-C.


Since α, β are independent of ℓ in Definition <ref>,
if (𝐀;𝒢) is EMSS-C, then there exist β≥ 1, α∈(0,1) such that
𝔼{x̅(k)^2}=∫_Θ𝔼{x̅(k)^2ϑ_0=ℓ}ν(ℓ)μ(dℓ)≤βα^kx_0^2∫_Θν(ℓ)μ(dℓ)=βα^kx_0^2,
which ends the proof.


The current idea of the proof of Proposition <ref> is acquired along the hint of an anonymous reviewer.


Below we attempt to discuss the EMSSy-C of (𝐀;𝒢) through operator 𝒯_A.
A necessary lemma is put forward first.

For any V={V(ℓ)}_ℓ∈Θ∈𝕊ℍ_∞^n and k∈ℕ,

    𝔼{Γ(k,0)'V(ϑ(k))Γ(k,0)|ϑ_0}=𝒯_A^k(V)(ϑ_0) a.s..



Let us prove the result by mathematical induction.
For any V∈𝕊ℍ_∞^n, if k=0, then (<ref>) becomes a tautology.
Suppose that (<ref>) holds for k=i∈ℕ^+.
According to Fubini's theorem, we get that

    𝔼{Γ(i+1,0)'V(ϑ(i+1))Γ(i+1,0)
    |ϑ_0}
    
    =   𝔼{𝔼[Γ(i,0)'A(ϑ(i))'
    V(ϑ(i+1))
    A(ϑ(i))Γ(i,0)|𝔉_i]|ϑ_0}
    
    =   𝔼{Γ(i,0)'A(ϑ(i))'
    𝔼[V(ϑ(i+1))|𝔉_i]
    A(ϑ(i))Γ(i,0)|ϑ_0}
    
    =   𝔼{Γ(i,0)'A(ϑ(i))'∫_ΘV(s)g(s|ϑ(i))μ(ds)
    A(ϑ(i))Γ(i,0)|ϑ_0}
    
    =   𝔼{Γ(i,0)'𝒯_A(V)
    (ϑ(i))Γ(i,0)|ϑ_0} a.s..

Recalling the hypothesis that (<ref>) holds for i, it follows from (<ref>) that
𝔼{Γ(i+1,0)'V(ϑ(i+1))Γ(i+1,0)
|ϑ_0}
=𝔼{Γ(i,0)'[𝒯_A(V)(ϑ(i))]
Γ(i,0)|ϑ_0}
=𝒯_A^i[𝒯_A(V)](ϑ_0)
=𝒯_A^i+1(V)(ϑ_0) a.s.,
i.e., (<ref>) is true for k=i+1.
Thus, (<ref>) holds for every k∈ℕ.


(𝐀;𝒢) is EMSS-C iff 𝒯_A generates an ESAE.


Let ϑ_0=ℓ∈Θ be arbitrary but fixed.
For any V∈𝕊ℍ_∞^n and k∈ℕ, by Lemma <ref>, we have that

    𝔼{Γ(k,0)'V(ϑ(k))Γ(k,0)
    |ϑ_0=ℓ}=𝒯_A^k(V)(ℓ) μ-a.e..

Letting V=ℐ in (<ref>), and pre- and post-multiplying by x_0' and x_0, respectively,
one can infer that for every k∈ℕ,
𝔼{x̅(k)^2|ϑ_0=ℓ}
=x_0'𝒯_A^k(ℐ)(ℓ)x_0, where x̅(k) is the system state of (𝐀;𝒢).

Sufficiency. If 𝒯_A generates an ESAE,
then there are β≥ 1, α∈(0,1) such that
𝒯_A^k(ℐ)(ℓ)≤𝒯_A^k≤βα^k, ∀ k∈ℕ.
Noticing that 𝒯_A^k(ℐ)(ℓ)∈𝕊^n, one can get that
𝔼{x̅(k)^2|ϑ_0=ℓ}≤x_0^2𝒯_A^k(ℐ)(ℓ).
Hence, we conclude that (𝐀;𝒢) is EMSS-C.

Necessity. If (𝐀;𝒢) is EMSS-C, then there are β_1≥ 1, α∈(0,1) such that
𝔼{x_0'Γ(k,0)'Γ(k,0)x_0|ϑ_0=ℓ}=𝔼{x̅(k)^2|ϑ_0=ℓ}≤β_1α^kx_0^2, ∀ k∈ℕ.
Then

    𝔼{Γ(k,0)'Γ(k,0)|ϑ_0=ℓ}≤β_1α^k, ∀ k∈ℕ.

Let V̂∈𝕊ℍ_∞^n be arbitrary and satisfy V̂_∞=1.
By Proposition <ref>,
there are V̂^+, V̂^-∈ℍ_∞^n+
such that V̂=V̂^+-V̂^- and max{V̂^+_∞, V̂^-_∞}≤1.
Thus, for every k∈ℕ, it is true that

    𝔼{Γ(k,0)'V̂(ϑ(k))
    Γ(k,0)|ϑ_0=ℓ}
    ≤   𝔼{Γ(k,0)'V̂^+(ϑ(k))
    Γ(k,0)|ϑ_0=ℓ}
       +𝔼{Γ(k,0)'V̂^-(ϑ(k))
    Γ(k,0)|ϑ_0=ℓ}.

Let V̅∈ℍ^n+_∞ be arbitrary and satisfy V̅_∞≤1.
Next, for every k∈ℕ, we will prove that

    𝔼{Γ(k,0)'V̅(ϑ(k))Γ(k,0)|ϑ_0=ℓ}
    ≤   𝔼{Γ(k,0)'Γ(k,0)|ϑ_0=ℓ}.

Since V̅_∞≤1, then for any z∈ℝ^n,
we make that z'V̅(ϑ(k))z≤ z' I_n× nz a.s.,
i.e., V̅(ϑ(k))≤ I_n× n a.s..
Hence,
𝔼{Γ(k,0)'V̅(ϑ(k))Γ(k,0)|ϑ_0=ℓ}≤𝔼{Γ(k,0)'Γ(k,0)|ϑ_0=ℓ}.
(<ref>) is then derived.
For every k∈ℕ, (<ref>)-(<ref>) result in
𝔼{Γ(k,0)'V̂(ϑ(k))Γ(k,0)|ϑ_0=ℓ}≤βα^k,
where β=2β_1. Combining this with (<ref>),
one can deduce that
𝒯_A^k(V̂)(ℓ)≤βα^k.
Therefore, 𝒯_A^k(V̂)_∞≤βα^k.
Then 𝒯_A^k≤βα^k is reached
since V̂∈𝕊ℍ_∞^n is arbitrary and V̂_∞=1.


By letting V=ℐ in (<ref>), and following a similar argument as the proof of Theorem <ref>,
we can prove the following corollary:

𝒯_A generates an ESAE iff there are β≥ 1, α∈(0,1) such that
𝒯_A^k(ℐ)(ℓ)≤βα^k μ-a.e. for every k ∈ℕ.


In the following theorem, the EMSSy of (𝐀;𝒢) is characterized by operator ℒ_A.

(𝐀;𝒢) is EMSS iff ℒ_A generates an ESCE.


For every k∈ℕ,
it follows from (ii) of Proposition <ref> that

    𝔼{x̅(k)^2}=⟨X̅(k); ℐ⟩=   ∫_Θtr{ℒ_A^k(X_0)(s)}μ (ds).

Sufficiency.
By Proposition <ref>, for every k∈ℕ, we have that
∫_Θtr{ℒ_A^k(X_0)(s)}μ (ds)≤ n∫_Θℒ_A^k(X_0)(s)μ (ds).
So, it can be made that 𝔼{x̅(k)^2}≤ nx_0^2ℒ_A^k.
Therefore, (𝐀;𝒢) is EMSS. 

Necessity.
By Proposition <ref>, one derives that
ℒ_A^k(X_0)_1 ≤∫_Θtr{ℒ_A^k(X_0)(s)}μ (ds).
From this fact, (<ref>), and the assumption that (𝐀;𝒢) is EMSS,
one can get that there are β≥ 1, α∈(0,1) such that

    ℒ_A^k(X_0)_1≤βα^kx_0^2≤ nβα^kX_0_1

holds for every k∈ℕ, x_0∈ℝ^n.
For any P∈𝕊ℍ_1^n, by Proposition <ref>, there exist P^+, P^-∈ℍ_1^n+ such that
P=P^+-P^- and P_1≥max{P^+_1,P^-_1}.
Therefore,
ℒ_A^k(P)_1≤ℒ_A^k(P^+)_1+ℒ_A^k(P^-)_1.
Next, we will show that for any P̅∈ℍ_1^n+, there are β_1≥ 1, α∈(0,1) such that

    ℒ_A^k(P̅)_1≤β_1α^kP̅_1,  ∀ k∈ℕ.

When P̅_1=0, (<ref>) is obviously trivial. Let P̅_1≠ 0
and set P̂={P̂(ℓ)}_ℓ∈Θ with P̂(ℓ)=P̅(ℓ) I_n× n.
It is obvious that P̅_1=P̂_1 and P̅≤P̂.
Hence 0 ≤ℒ_A^k(P̅) ≤ℒ_A^k(P̂), which yields that
ℒ_A^k(P̅)_1≤ℒ_A^k(P̂)_1.
On the other hand, P̂ can be written as P̂=∑_i=1^nP̂_i
with P̂_i={P̂_i(ℓ)}_ℓ∈Θ,
where P̂_i(ℓ)=P̅(ℓ)e_ie_i' with e_i'=(0,0,⋯,0,1,0,⋯,0)
(e_i∈ℝ^n is the vector where the i-th term is 1 and the rest is 0).
Now we arrive at

    ℒ_A^k(P̂)_1≤∑_i=1^nℒ_A^k(P̂_i)_1, ∀ k∈ℕ.

As for P̂_i, i∈1,n, notice that P̂_i_1=P̅_1.
In addition, there is ν∈ℍ_1^n+ given by ν(ℓ)=P̅(ℓ)/P̅_1
and x_0^i=√(P̅_1)e_i∈ℝ^n
such that P̂_i=X_0^i with X_0^i(ℓ)=x_0^i (x_0^i)'ν(ℓ).
So, one can rewrite (<ref>) as
ℒ_A^k(P̂)_1 ≤∑_i=1^nℒ_A^k(X_0^i)_1.
Moreover, it can be directly drawn that X_0^i_1=P̂_i_1=P̅_1.
Then, according to (<ref>), for every i∈1,n, there are β≥ 1, α∈(0,1) such that
ℒ_A^k(X_0^i)_1≤ nβα^kP̅_1
holds for every k ∈ℕ, x_0^i∈ℝ^n.
Further, ℒ_A^k(P̅)_1≤ n^2βα^kP̅_1,
i.e.,
(<ref>) is proved with β_1=n^2β.
Consequently,
ℒ_A^k(P^+)_1≤β_1α^kP^+_1≤β_1α^kP_1
and ℒ_A^k(P^-)_1≤β_1α^kP_1, ∀ k∈ℕ.
Now we conclude that there are β_2=2 n^2β≥ 1, α∈(0,1) such that
ℒ_A^k(P)_1≤β_2α^kP_1
holds for every k ∈ℕ, P∈𝕊ℍ_1^n.
Thus, ℒ_A generates an ESCE.


Noticing the implications that 𝒯_A (ℒ_A) generates an ESAE (ESCE)
iff r_σ(𝒯_A)<1 (r_σ(ℒ_A)<1)(see <cit.>, page 33),
from Theorems <ref> and <ref> we can make the following theorem,
which provides relatively complete and unified methods for determining EMSSy-C and EMSSy of (𝐀;𝒢), respectively.

(𝐀;𝒢) is EMSS-C iff r_σ(𝒯_A)<1;
(𝐀;𝒢) is EMSS iff r_σ(ℒ_A)<1.


Consider (𝐀;𝒢).
When specializing Theorem <ref> to finite MJLSs, EMSSy-C is equivalent to EMSSy.
Indeed, in this finite case, 𝒯_A coincides with the adjoint ℒ_A^* of ℒ_A.
Hence, r_σ(ℒ_A)<1 iff r_σ(𝒯_A)<1,
and the equivalence between these two kinds of stabilities of (𝐀;𝒢) can be gained by Theorem <ref>.
The same result in the continuous-time case for finite MJLSs could be consulted in Theorem 3.2.4 of <cit.> (page 135).
However, for the case where the Markov chain takes values in Θ,
establishing the equivalence between these two kinds of stabilities is not a trivial task.
Recalling Proposition <ref>,
it shows that (𝐀;𝒢) is EMSS provided (𝐀;𝒢) is EMSS-C.
We conjecture that the converse result still holds under the assumptions considered in this paper.
This problem is challenging but meaningful, and requires a more thorough discussion.



(𝐀;𝒢) is EMSS iff (𝐀;𝒢) is EMSS-C.


In the following, we will present the equivalent conditions for (𝐀;𝒢) to be EMSS-C
in terms of the existence of uniformly positive definite solutions of coupled Lyapunov-type equations or inequalities in Banach spaces.

The following assertions are equivalent:

(i) (𝐀;𝒢) is EMSS-C; 

(ii) Given any V∈ℍ_∞^n+*, there exists U∈ℍ_∞^n+* such that

    U(ℓ)-𝒯_A(U)(ℓ)=V(ℓ) μ-a.e.;

(iii) There exists U∈ℍ_∞^n+* such that
U(ℓ)-𝒯_A(U)(ℓ)= I_n× n μ-a.e.;

(iv) There is U∈ℍ_∞^n+* and ξ>0 such that
U(ℓ)-𝒯_A(U)(ℓ)≥ξ I_n× n μ-a.e..


First, we prove (i)→ (ii).
It follows from Theorem <ref> that r_σ(𝒯_A)<1 when (𝐀;𝒢) is EMSS-C.
Hence, for any V∈ℍ_∞^n+*, U=(ℐ_𝕊ℍ_∞^n-𝒯_A)^-1(V)
=∑_k=0^∞𝒯_A^k(V)∈ℍ_∞^n+* satisfies (<ref>).
The implications (ii)→ (iii)→ (iv) can be got from
the standard arguments concerning the bounded linear operator on Banach space ℍ_∞^n, and the details are omitted.
Now we prove (iv)→ (i).
Set V=U-𝒯_A(U).
If (iv) holds, then δ I_n× n> U(ℓ)≥ξ I_n× n μ-a.e., where δ>U_∞.
Let α=1-ξ/δ. Obviously, α∈(0,1).
Next, for every k∈ℕ^+, we will show that

    𝒯_A^k(U)(ℓ)≤α^kU(ℓ)

holds by mathematical induction.
For k=1, since U(ℓ)-𝒯_A(U)(ℓ)≥ξ I_n× n>ξ/δU(ℓ),
it implies that (<ref>) holds for k=1.
We assume that (<ref>) is true for k=i∈ℕ^+.
Considering that 𝒯_A is a positive operator on 𝕊ℍ_∞^n,
thus
𝒯_A^i(U)(ℓ)-𝒯_A^i+1(U)(ℓ)
≥ξ𝒯_A^i(ℐ)(ℓ)≥ξ/δ𝒯_A^i(U)(ℓ).
Combining this with the assumption of (<ref>) holding for k=i,
we get that 𝒯_A^i+1(U)(ℓ)≤α𝒯_A^i(U)(ℓ)
≤α^i+1U(ℓ).
Therefore, for every k∈ℕ^+, (<ref>) is valid.
So, it can be confirmed that
ξ𝒯_A^k(ℐ)(ℓ)≤𝒯_A^k(U)(ℓ)≤α^kU(ℓ)
≤δα^kℐ(ℓ).
This gives rise to 𝒯_A^k(ℐ)(ℓ)≤βα^kI_n× n,
where β=δ/ξ>1.
Further, we have that 𝒯_A^k(ℐ)(ℓ)≤βα^k,
and (i) is then inferred according to Corollary <ref> and Theorem <ref>.


If (𝐀;𝒢) is EMSS-C, then there exists
U∈ℍ^n+_∞ such that

    U(ℓ)-𝒯_A(U)(ℓ)=C(ℓ)'C(ℓ) μ-a.e..



Following the same reasoning as in Theorem <ref> for the implication (i)→ (ii), the desired result can be proved.
It is worth noting that the positive semidefinite term C(ℓ)'C(ℓ) only guarantees that the solution of (<ref>) satisfies U∈ℍ_∞^n+.




§ BOUNDED REAL LEMMA

In this section, we aim to develop the H_∞ theory for the perturbation attenuation analysis of Φ_v,
where v is an exogenous perturbation with finite energy.
Both the finite and infinite horizon cases are included.
Now we start by introducing the following sets:
For T∈ℕ,
l^2(0,T;ℝ^r)={v|v={v(k)}_k∈0,T
is an ℝ^r-valued random variables sequence
with that v(k) is 𝔉_k-measurable, ∀ k∈0,T,
and v_l^2(0,T;ℝ^r):
=(∑_k=0^T𝔼{v(k)^2})^1/2<∞}.
For any ℓ∈Θ,
l^2_ℓ(0,T;ℝ^r)
={v|v={v(k)}_k∈0,T is an ℝ^r-valued random variables sequence
with that v(k) is 𝔉_k-measurable, ∀ k∈0,T, and
v_l^2_ℓ(0,T;ℝ^r)=:(∑_k=0^T
𝔼{v(k)^2|ϑ_0=ℓ})^1/2<∞};
l^2_ℓ(ℕ;ℝ^r)
={v|v={v(k)}_k∈ℕ
is an ℝ^r-valued random variables sequence with that v(k) is 𝔉_k-measurable, ∀ k∈ℕ,
and v_l^2_ℓ(ℕ;ℝ^r)
:=(∑_k=0^∞
𝔼{v(k)^2|ϑ_0=ℓ})^1/2<∞}.


(i) l^2(0,T;ℝ^r) is a Hilbert space with the norm
·_l^2(0,T;ℝ^r)
induced by the usual inner product;

(ii) For any ℓ∈Θ, l^2_ℓ(0,T;ℝ^r) and l^2_ℓ(ℕ;ℝ^r)
are Hilbert spaces with
·_l^2_ℓ(0,T;ℝ^r) and ·_l^2_ℓ(ℕ;ℝ^r), respectively.


Let l^2(0,T;ℝ^r) be the space of all r-dimensional measurable and square-summable random sequences
v(·) : 0,T×Ω→ℝ^r.
Following a similar argument as in <cit.>,
it makes known that l^2(0,T;ℝ^r) is a Hilbert space
with the norm v_2=(∑_k=0^T𝔼{v(k)^2})^1/2.
Therefore, l^2(0,T;ℝ^r) is a Hilbert space with ·_l^2(0,T;ℝ^r)
owing to being a closed subspace of l^2(0,T;ℝ^r).
For any ℓ∈Θ, it can be shown that l^2_ℓ(0,T;ℝ^r)
and l^2_ℓ(ℕ;ℝ^r) are also Hilbert spaces similarly.


For convenience, v∈ l^2_Θ(0,T;ℝ^r) means that v∈ l^2_ℓ(0,T;ℝ^r)
and v_l^2_ℓ(0,T;ℝ^r)<∞ μ-a.e.; v∈ l^2_Θ(ℕ;ℝ^r) means that
v∈ l^2_ℓ(ℕ;ℝ^r) and v_l^2_ℓ(ℕ;ℝ^r)<∞ μ-a.e..

In what follows, to avoid confusion, we represent the state and output response of Φ_v
as the generalized functions of the initial conditions:
x(·)=ϕ_x(·,x_0,ϑ_0,v) is the state response of Φ_v
with the initial conditions ϑ(0)=ϑ_0 and x(0)=x_0;
y(·)=ϕ_y(·,x_0,ϑ_0,v)=C(ϑ(·))ϕ_x(·,x_0,ϑ_0,v)+D(ϑ(·))v(·)
is the corresponding output response.
Given γ>0, for U∈ℍ_∞^n, define

    Ψ_1(U)(ℓ)=𝒯_A(U)(ℓ)+C(ℓ)'C(ℓ),
                Ψ_2(U)(ℓ)=A(ℓ)'ℰ(U)(ℓ)B(ℓ),
    
                Ψ_3^γ(U)(ℓ)=𝒯_B(U)(ℓ)+D(ℓ)'D(ℓ)-γ^2 I_r× r,
                ℱ(U)(ℓ)=-Ψ_3^γ(U)(ℓ)^-1Ψ_2(U)(ℓ)',  ℓ∈Θ,

where 𝒯_B is given by (<ref>) with K=B.



 §.§ The finite horizon case

In this subsection, the finite horizon BRL, given in Theorem <ref>,
presents a sufficient and necessary condition to ensure that the finite horizon H_∞ norm of the linear perturbation operator
is below a prescribed level γ>0.

Consider Φ_v.
For T∈ℕ, if v∈ l^2(0,T;ℝ^r), then
{ϕ_y(k,0,ϑ_0,v)}_k∈0,T∈ l^2(0,T;ℝ^m).
Hence, we could define the linear perturbation operator
𝕃_T:l^2(0,T;ℝ^r)→ l^2(0,T;ℝ^m)
with 𝕃_T(v)(k)=ϕ_y(k,0,ϑ_0,v), ∀ k∈0,T,
and its H_∞ norm 𝕃_T is

    𝕃_T:=sup_v∈ l^2(0,T;ℝ^r),
     v≠0,  x_0=0𝕃_T(v)_l^2(0,T;ℝ^m)/v_l^2(0,T;ℝ^r).

𝕃_T can measure the effect of the unknown disturbance v on the output {ϕ_y(k,0,ϑ_0,v)}_k∈0,T
in the worst-case scenario,
and a larger value of 𝕃_T indicates a greater effect of v on {ϕ_y(k,0,ϑ_0,v)}_k∈0,T.
Therefore, finding a way to make 𝕃_T smaller than a prescribed level is an important issue
in analyzing the disturbance attenuation properties of Φ_v.

Given γ>0 and T∈ℕ, for any x_0∈ℝ^n, define the cost functionals

    J_γ(T,x_0,v):=∑_k=0^T𝔼{y(k)^2-γ^2v(k)^2},
                                         ∀ v∈ l^2(0,T;ℝ^r);
          J_γ(T,x_0,ℓ,v):=∑_k=0^T𝔼{y(k)^2-γ^2v(k)^2|ϑ_0=ℓ},
                                         ∀ v∈ l^2_ℓ(0,T;ℝ^r), ∀ℓ∈Θ.


The following propositions state some properties of 𝕃_T and the cost functionals.
The proofs are relatively simple and therefore omitted.

Given γ>0 and T∈ℕ, for any v∈ l^2_Θ(0,T;ℝ^r), J_γ(T,x_0,v)=∫_ΘJ_γ(T,x_0,ℓ,v)ν(ℓ)μ(dℓ).


Given γ>0 and T∈ℕ, for any v∈ l^2(0,T;ℝ^r)
with v≠ 0,
J_γ(T,0,v)≤0 iff 𝕃_T≤γ.
Moreover,
J_γ(T,0,v)<0 iff 𝕃_T<γ.


Given γ>0 and T∈ℕ, for every k∈0,T, set F(k)={F(k,ℓ)}_ℓ∈Θ∈ℍ_∞^r× n.
Consider the backward difference equation:

    {[ Y (k,ℓ )=[A (ℓ )+B (ℓ )F (k,ℓ )]'ℰ (Y (k+1 ) ) (ℓ );       ·[A (ℓ )+B (ℓ )F (k,ℓ )]-γ^2F (k,ℓ )'F (k,ℓ );             +[C(ℓ)+D(ℓ)F(k,ℓ)]'[C(ℓ )+D(ℓ )F(k,ℓ)],;                        Y (T+1,ℓ )=0,  ∀ k∈0,T, ℓ∈Θ. ].

Obviously, the solution of (<ref>) can be obtained by iteration and is denoted by
Y_T^F,γ(k)={Y_T^F,γ(k,ℓ)}_ℓ∈Θ∈ℍ_∞^n, k∈0,T+1.
In addition, we can rewrite (<ref>) as

    (
                           [ I_n× n; F(k,ℓ);        ])'
    W(Y_T^F,γ,k,ℓ)
    (
      [ I_n× n; F(k,ℓ);        ])=0,

where

W(Y,k,ℓ)
=([ Ψ_1(Y(k+1))(ℓ)-Y(k)(ℓ)         Ψ_2(Y(k+1))(ℓ);        Ψ_2(Y(k+1))(ℓ)'  Ψ_3^γ(Y
    (k+1))(ℓ);                        ]).

The following lemma will be used to assist in proving the necessity part of the finite horizon BRL (Theorem <ref>).

Fix γ>0 and T∈ℕ.
Set F(k)∈ℍ_∞^r× n, k∈0,T.
If Φ_v satisfies 𝕃_T<γ, then (<ref>) admits a solution such that

    Ψ_3^γ(Y_T^F,γ(k+1))(ℓ)≤-η_0 I_r× r μ-a.e.,  ∀η_0∈(0,γ^2-𝕃_T^2)

holds for every k∈0,T.


The proof of this lemma will be divided into two steps.
First, we will prove that

    Ψ_3^γ(Y_T^F,γ(k+1))(ℓ)≤0 μ-a.e.,
    ∀ k∈0,T.

The second step, following a similar argument as the proof of Proposition 8.4 in <cit.>,
can use (<ref>) to show (<ref>).
Therefore, here we just have to prove (<ref>).
Since Φ_v satisfies 𝕃_T<γ, by Proposition <ref>,
for any v∈ l^2(0,T;ℝ^r) with v≠ 0, we have that J_γ(T,0,v)<0.
Now let us assume that (<ref>) does not hold,
then there is k_0∈0,T and Δ∈ℬ(Θ) with μ(Δ)>0
such that for any ℓ_Δ∈Δ, there exists κ(ℓ_Δ)∈ℝ^r
satisfying κ(ℓ_Δ)=1 and that

    κ(ℓ_Δ)'
    Ψ_3^γ(Y_T^F,γ(k_0+1))
    (ℓ_Δ)
    κ(ℓ_Δ)>0

holds μ-a.e. on Δ.
For every k∈0,T,
define v̅(k)∈ l^2(0,T;ℝ^r) by
v̅(k)={κ(ϑ(k)) ·χ_{ϑ(k)∈Δ},    k=k_0,  

0,         k≠ k_0,
.
in which χ_{·} is an indicator function.
Set v̂(k)=F(k,ϑ(k))z(k)+v̅(k), k∈0,T, where z(k)=ϕ_x(k,0,ϑ_0,v̂).
Clearly, v̂∈ l^2(0,T;ℝ^r) and v̂≠ 0.
Noticing that Y_T^F,γ(k), k∈0,T+1 is the solution of (<ref>),
we calculate 𝔼{[z(k+1)'Y^F,γ_T(k+1)(ϑ(k+1))z(k+1) -z(k)'Y^F,γ_T(k)(ϑ(k))z(k)]|ϑ_0=ℓ}
and sum it over k, then

    0=   ∑_k=0^T𝔼{[(
                           [  z(k); v̂(k);       ])'
    W(Y^F,γ_T,k,ϑ(k))
                  (
                           [  z(k); v̂(k);       ])
                 -v̂(k)'(D(ϑ(k))'D(ϑ(k))
    -γ^2 I_r× r)v̂(k)
                 -z(k)'C(ϑ(k))'C(ϑ(k))z(k)]|ϑ_0=ℓ}.

Considering (<ref>) and (<ref>), one can rewrite (<ref>) for Φ_v̂ with x_0=0 as follows:

    J_γ(T,0,ℓ,v̂)
    
    =   ∑_k=0^T𝔼{v̅(k)'Ψ_3^γ(Y_T^F,γ(k+1))
    (ϑ(k))v̅(k)
              +2z(k)'[ Ψ_3^γ(Y_T^F,γ(k+1))(ϑ(k))F(k,ϑ(k))
              +Ψ_2(Y_T^F,γ(k+1))(ϑ(k))]v̅(k)|ϑ_0=ℓ}.

Since v̅(k)=0 for k≠ k_0 and z(k) is the state response of Φ_v̂ with x_0=0,
z(k)=0 for k≤ k_0 is concluded.
So, (<ref>) can be rewritten as
J_γ(T,0,ℓ,v̂)=𝔼{v̅(k_0)'Ψ_3^γ(Y_T^F,γ(k_0+1))
(ϑ(k_0))v̅(k_0)|ϑ_0=ℓ}.
In view of this, by Proposition <ref>, we have that
J_γ(T,0,v̂)=∫_Θ
𝔼{κ(ϑ(k_0))'
Ψ_3^γ(Y_T^F,γ(k_0+1))
(ϑ(k_0))
κ(ϑ(k_0))
·χ_{ϑ(k_0)∈Δ}
|ϑ_0=ℓ}ν(ℓ)μ(dℓ).
Further, J_γ(T,0,v̂)> 0
follows from (<ref>) and Assumptions <ref> and <ref>,
which is a contradiction with that for any v∈ l^2(0,T;ℝ^r) and v≠ 0, J_γ(T,0,v)<0.
(<ref>) is therefore proved.


When the Markov chain takes values in a countable set, to prove (<ref>),
it only needs to assume the existence of a discrete point l_0∈Θ satisfying (<ref>)
to lead to a contradiction (refer to the proof of Proposition 8.4 in <cit.>).
However, as mentioned earlier, the Borel space considered in this paper is a general state space,
including the case of continuous state space, so the above method is no longer applicable,
and it requires to utilize measure theory to derive Lemma <ref>.


The result below is immediate from Lemma <ref>.

Given γ>0, if there exists T∈ℕ^+ such that Φ_v satisfies 𝕃_T<γ,
then D(ℓ)'D(ℓ)-γ^2 I_r× r<0 μ-a.e..


Now we are prepared to propose the finite horizon BRL, which is the main result of this subsection.

Given γ>0 and T∈ℕ, Φ_v satisfies 𝕃_T<γ iff the following backward difference equation:

    {[                              Y(k,ℓ)=        Ψ_1(Y(k+1))(ℓ)-Ψ_2(Y(k+1))(ℓ);                                      ·Ψ_3^γ(Y(k+1))(ℓ)^-1Ψ_2(Y(k+1))(ℓ)',;                               Y(T+1,                   ℓ)=0, ∀ k∈0,T, ℓ∈Θ;                                      ].

admits a solution Y_T^γ(k)∈ℍ_∞^n+ such that

    Ψ_3^γ(Y_T^γ(k+1))(ℓ)≤-η_0 I_r× r μ-a.e., ∀η_0∈(0,γ^2-𝕃_T^2),

for every k∈0,T.


Sufficiency.
If (<ref>) admits a solution Y_T^γ(k)∈ℍ_∞^n+, k∈0,T, then

    J_γ(T,x_0,v)
    =      ∑_k=0^T𝔼{([ x(k); v(k);      ])'
                                W(Y_T^γ,k,ϑ(k))
                                ([ x(k); v(k);      ])}
          +𝔼{x_0'Y_T^γ(ϑ_0)x_0},

where x(·) is the system state of Φ_v.
Since Y_T^γ(k), k∈0,T is the solution of (<ref>), then by letting x_0=0,
(<ref>) can be rewritten as
J_γ(T,0,v)=∑_k=0^T𝔼{[v(k)-v̂(k)]' ·Ψ_3^γ(Y_T^γ(k+1))(ϑ(k))
[v(k)-v̂(k)]}≤ 0,
in which v̂(k)=ℱ(Y_T^γ(k+1))(ϑ(k))ϕ_x(k,0,ϑ_0,v).
Combining this with (<ref>), one obtains that J_γ(T,0,v)=0 iff v(k)=v̂(k), ∀ k∈0,T.
Now substituting v̂(k) into Φ_v with x_0=0, we deduce that
ϕ_x(k,0,ϑ_0,v̂)≡ 0, which implies that for every k∈0,T, v̂(k)=0.
Hence, if v(k)≠0, then
J_γ(T,0,v)≤-η_0v-v̂^2
_l^2(0,T;ℝ^r)<0.
Therefore, 𝕃_T<γ is concluded by Proposition <ref>.

Necessity.
For k=T, by substituting F(k,ℓ)=ℱ(Y_T^γ(k+1))(ℓ) into (<ref>),
we can get that the obtained equation is consistent with (<ref>).
Moreover, Y_T^γ(T, ℓ)=C(ℓ)'C(ℓ)≥ 0.
Using the assumption of 𝕃_T<γ and Lemma <ref>, it follows that
(<ref>) is satisfied for k=T.
Next, (<ref>) is iteratively proceeded backwards, starting from T-1 and stopping at k=0.
For each k∈{T-1,T-2,⋯,0}, repeating the process as the case of k=T,
we assert that (<ref>) admits a solution Y_T^γ(k)∈ℍ_∞^n+ satisfying (<ref>).




 §.§ The infinite horizon case


In this subsection, we will give the infinite horizon BRL,
which can characterize internal stability and the H_∞ performance of MJLSs
through the existence of the stabilizing solution of the Θ-coupled ARE.
Before that, we provide the following definition,
which generalizes the notion of input-to-state stability to the system considered in this paper:

We call that Φ_v is input-to-state stable with conditioning, if for any x_0∈ℝ^n,
x={x(k)}_k∈ℕ∈ l^2_Θ(ℕ;ℝ^r) whenever v∈ l^2_Θ(ℕ;ℝ^r),
where x(·) is the system state of Φ_v.


For the MJLS where the Markov chain takes values in a countably infinite set,
it was shown in <cit.> that EMSSy-C of the autonomous system can guarantee the input-to-state stability of Φ_v,
which is not achievable with stochastic stability.
The following proposition can be regarded as a generalization of Theorem 3.3 in <cit.>.
Its proof is similar to that of Theorem 3.3 and thus is omitted.

If (𝐀;𝒢) is EMSS-C, then for any ℓ∈Θ,
x={x(k)}_k∈ℕ∈ l^2_ℓ(ℕ;ℝ^n) whenever v∈ l^2_ℓ(ℕ;ℝ^r),
where x(·) is the system state of Φ_v.
Furthermore, Φ_v is input-to-state stable with conditioning provided (𝐀;𝒢) is EMSS-C.


Note that Proposition <ref> yields the next corollary:

If (𝐀;𝒢) is EMSS-C,
then we have that lim_k→∞𝔼{x(k)^2|ϑ_0=ℓ}=0 μ-a.e.
whenever ∑_k=0^∞𝔼{v(k)^2|ϑ_0=ℓ}<∞ μ-a.e.,
where x(·) is the system state of Φ_v.


We need to emphasize that all the analysis below is under the assumption that Φ_v is internally stable,
i.e., (𝐀;𝒢) is EMSS-C.

According to Proposition <ref>, for any ℓ∈Θ, if v∈ l^2_ℓ(ℕ;ℝ^r),
then {ϕ_x(k,0,ℓ,v)}_k∈ℕ∈ l^2_ℓ(ℕ;ℝ^n)
and a straightforward calculation yields {ϕ_y(k,0,ℓ,v)}_k∈ℕ∈ l^2_ℓ(ℕ;ℝ^m).
Therefore, for every k∈ℕ, we can define the linear perturbation operator
𝔏_∞(v)(k,ℓ):=ϕ_y(k,0,ℓ,v)
and its H_∞ norm:

    𝔏_∞:   
    =_ℓ∈Θ, v∈ l^2_ℓ(ℕ;ℝ^r),
     v≠0, x_0=0𝔏_∞(v)_l^2_ℓ(ℕ;ℝ^m)/v_l^2_ℓ(ℕ;ℝ^r).


Assumption 2 is necessary to ensure that v_l^2_ℓ(ℕ;ℝ^r)
=(∑_k=0^∞𝔼{v(k)^2|ϑ_0=ℓ})^1/2>0  μ-a.e. on Θ
and (<ref>) is well-defined.
Indeed, if Assumption 2 does not hold, then there exists Δ_0∈ℬ(Θ) with μ(Δ_0)>0 such that μ̂(ϑ_0∈Δ_0)=∫_Δ_0ν(ℓ)μ(dℓ)=0,
where μ̂ is the distribution of ϑ_0.
This implies that ∑_k=0^∞𝔼{v(k)^2|ϑ_0=ℓ}=0 μ-a.e. on Δ_0.
One can refer to Remark 8 in <cit.> for the case of finite MJLSs.


Given γ>0, and letting T→∞ in (<ref>),
for any v∈ l^2_Θ(ℕ;ℝ^r),
we have the corresponding form for the infinite horizon case denoted by  J_γ(∞,x_0,ℓ,v).
Similar to the scenario of the finite horizon,
the following proposition on 𝔏_∞ and J_γ(∞,x_0,ℓ,v) can be shown.

Given γ>0, for any v∈ l^2_Θ(ℕ;ℝ^r) with v≠ 0, J_γ(∞,0,ℓ,v)≤0 μ-a.e.
iff 𝔏_∞≤γ.
Moreover, J_γ(∞,0,ℓ,v)<0 μ-a.e. iff 𝔏_∞<γ.


The following job is to apply the finite horizon BRL developed in Subsection <ref> to work out the infinite horizon scenario.
First, we present two necessary propositions.

For every T∈ℕ, 𝕃_T≤𝔏_∞.


For every T∈ℕ and ℓ∈Θ, if v∈ l^2_ℓ(0,T;ℝ^r),
then {ϕ_y(k,0,ℓ,v)}_k∈0,T∈ l^2_ℓ(0,T;ℝ^m).
Therefore, we can give the linear perturbation operator
𝔏_T(v)(k,ℓ)=ϕ_y(k,0,ℓ,v), ∀ k∈0,T
and its  H_∞ norm:

    𝔏_T:   
    =_ℓ∈Θ,   v∈ l^2_ℓ(0,T;ℝ^r),
    
    v≠0, x_0=0𝔏_T(v)_l^2_ℓ
    (0,T;ℝ^m)/v_l^2_ℓ(0,T;ℝ^r).

Hence, for any v∈ l^2_Θ(0,T;ℝ^r),
𝔏_T·∑_k=0^T𝔼{v(k)^2
|ϑ_0=ℓ}≥∑_k=0^T𝔼{ϕ_y(k,0,ϑ_0,v)^2|ϑ_0=ℓ} μ-a.e..
According to Assumption <ref> and Theorem 15.2 in <cit.>,
it follows that for any v∈ l^2(0,T;ℝ^r),
∑_k=0^T𝔼{ϕ_y(k,0,ϑ_0,v)^2}≤𝔏_T·∑_k=0^T𝔼{v(k)^2}.
Further, bearing in mind (<ref>), we have that 𝕃_T≤𝔏_T.
Next, we will prove 𝔏_T≤𝔏_∞.
For any v∈ l^2_ℓ(0,T;ℝ^r), ℓ∈Θ,
setting v̅={v̅(k)}_k∈ℕ∈ l^2_ℓ(ℕ;ℝ^r) with
v̅(k)={
v(k),    ∀ k∈0,T,  

0,       otherwise,
.
one can get that

    ∑_k=0^T𝔼{ϕ_y(k,0,ϑ_0,v)^2
    |ϑ_0=ℓ}/∑_k=0^T𝔼{v(k)^2
    |ϑ_0=ℓ}
    
    =   ∑_k=0^∞𝔼{ϕ_y(k,0,ϑ_0,v̅)^2
    |ϑ_0=ℓ}/∑_k=0^∞𝔼{v̅(k)^2
    |ϑ_0=ℓ}≤𝔏_∞,

which leads to 𝔏_T≤𝔏_∞.
Therefore, 𝕃_T≤𝔏_∞ is carried out from 𝕃_T≤𝔏_T.


Given γ>0,
for any x_0∈ℝ^n and v∈ l^2_Θ(ℕ;ℝ^r), if Φ_v is internally stable and satisfies 𝔏_∞<γ,
then
there exists
ζ>0 such that J_γ(∞,x_0,ℓ, v) ≤ζx_0^2 μ-a.e..


Let T∈ℕ be arbitrary but fixed.
If Φ_v is internally stable,
then by Theorem <ref>,
there is U∈ℍ^n+_∞ satisfying U-𝒯_A(U)=C'C.
This leads to

    𝔼{x(T+1)'U(ϑ(T+1))
    x(T+1)|ϑ_0=ℓ}
    
    =   x_0'U(ℓ)x_0+∑_k=0^T𝔼{
    -x(k)'C(ϑ(k))'C(ϑ(k))x(k)
    
    +   2x(k)'Ψ_2(U)(ϑ(k))v(k) +v(k)'𝒯_B(U)(ϑ(k))v(k)
    |ϑ_0=ℓ},

where x(·) is given by Φ_v with x(0)=x_0.
For any v∈ l^2_Θ(ℕ;ℝ^r), according to Corollary <ref>, one has that
lim_T→∞𝔼{x(T)'U(ϑ(T))x(T)|ϑ_0=ℓ}=0.
Letting T→∞ in (<ref>), it follows that
J_γ(∞,x_0,ℓ, v)
=∑_k=0^∞𝔼{[2x(k)'
Ψ_2(U)(ϑ(k))v(k)+v(k)'Ψ_3^γ(U)(ϑ(k))v(k)] |ϑ_0=ℓ}+x_0'U(ℓ)x_0.
Then,
J_γ(∞,x_0,ℓ, v)-J_γ(∞,0,ℓ, v)
=x_0'U(ℓ)x_0
+∑_k=0^∞𝔼{2x̅(k)'
Ψ_2(U)(ϑ(k)) v(k)|ϑ_0=ℓ},
where x̅(k) is the system state of (𝐀;𝒢).
Considering that 𝔏_∞<γ,
and taking ε with 𝔏_∞^2≤γ^2-ε^2,
J_γ(∞,0,ℓ,v)≤-ε^2∑_k=0^∞𝔼{v(k)^2|ϑ_0=ℓ}
can be concluded from Proposition <ref>.
So,
J_γ(∞,x_0,ℓ, v)
≤1/ε^2∑_k=0^∞𝔼{Ψ_2(U)(ϑ(k))
x̅(k)^2
|ϑ_0=ℓ}+x_0'U(ℓ)x_0+Π,
where Π=-∑_k=0^∞𝔼{ε v(k)-1/εΨ_2(U)(ϑ(k))
x̅(k)^2|ϑ_0=ℓ}≤0.
Since (𝐀;𝒢) is EMSS-C, there are β≥ 1, α∈(0,1) such that
∑_k=0^∞𝔼{x̅(k)^2| ϑ_0=ℓ}≤β/1-αx_0^2.
Setting ζ with ζ>U_∞+β/ε^2(1-α)U_∞^2A_∞^2B_∞^2≥0,
J_γ(∞,x_0,ℓ, v)≤ζx_0^2 is then fulfilled.


Given γ>0, for every T∈ℕ and k∈0,T+1, define
K_T^γ(k)={K_T^γ(k,ℓ)}_ℓ∈Θ by

    K_T^γ(k,ℓ)=Y_T^γ(T+1-k,ℓ), ℓ∈Θ,

where Y_T^γ(·) is the solution of (<ref>).
Obviously, K_T^γ(0,ℓ)=0, ℓ∈Θ.
And K_T^γ(k), k∈ℕ solves the following difference equation:

    K^γ(k+1,ℓ)=   Ψ_1(K^γ(k))(ℓ)
    -Ψ_2(K^γ(k))(ℓ)
       ·Ψ_3^γ(K^γ(k))(ℓ)^-1Ψ_2(K^γ(k))(ℓ)'.

In addition, for every T_1∈ℕ^+ and T_2∈ℕ^+,
one can get that K^γ_T_1(k)=K^γ_T_2(k), ∀ k∈0,min{T_1,T_2}.
Therefore, we could write K^γ(k)={K^γ(k,ℓ)}_ℓ∈Θ, ∀ k∈ℕ
as the solution of (<ref>) with K^γ(0,ℓ)=0, ℓ∈Θ.

In Theorem <ref>, the finite horizon BRL was set up based on the solution of (<ref>).
It is noticeable that (<ref>) is the bridge to connect (<ref>) with (<ref>).
The following lemma confirms the existence of the solution of (<ref>).

Given γ>0, if Φ_v is internally stable and satisfies 𝔏_∞<γ,
then (<ref>) admits a solution K^γ(k)∈ℍ_∞^n+, k∈ℕ
with the initial condition K^γ(0,ℓ)=0, ℓ∈Θ
such that

(i) Ψ_3^γ(K^γ(k))(ℓ)≤-η_0 I_r× r μ-a.e., ∀η_0∈(0,γ^2-𝔏_∞^2); 

(ii) 0≤ K^γ(k,ℓ)≤ K^γ(k+1,ℓ)≤ζ I_n× n μ-a.e., where ζ>0.


If 𝔏_∞<γ,
then by Proposition <ref>, 𝕃_T<γ for every T∈ℕ.
Let T∈ℕ be arbitrary but fixed.
Applying Theorem <ref>,
Y_T^γ(h)∈ℍ_∞^n+, ∀ h∈0,T is well-defined as the solution of (<ref>).
Then it concludes through (<ref>) that
(<ref>) admits a solution K^γ(k)∈ℍ_∞^n+, k∈ℕ
with the initial condition K^γ(0,ℓ)=0, ℓ∈Θ,
and (i) is proved.
Next, we will prove (ii).
By a similar proof with Proposition 8.5 in <cit.>,
we have that K^γ(k)≤ K^γ(k+1), ∀ k∈ℕ.
Consider v_T={v_T(k̅)}_k̅∈0,T∈ l^2_Θ(0,T;ℝ^r)
defined by v_T(k̅)=ℱ(Y_T^γ(k̅+1))(ϑ(k̅))x_T(k̅),
where x_T(k̅) is the solution of

    {[ x(k̅+1)=[A(ϑ(k̅))
    +B(ϑ(k̅))ℱ(Y_T^γ(k̅+1))
    (ϑ(k̅))]x(k̅),;                                                x(0)=x_0.;                                                          ].

And set v̂_T={v̂_T(k)}_k∈ℕ∈ l^2_Θ(ℕ;ℝ^r) by
v̂_T(k)={
v_T(k),     ∀ k∈0, T,  

0,      otherwise.
.
So for any x_0∈ℝ^n,
there exists ζ>0 satisfying x_0' Y_T^γ(0, ℓ) x_0
=J_γ(T,x_0,ℓ,v_T)≤ J_γ(∞,x_0,ℓ, v̂_T) ≤ζx_0^2,
in which the last inequality is got from Proposition <ref>.
Therefore, because of (<ref>) and the arbitrary of T∈ℕ,
K^γ(k+1,ℓ)≤ζ I_n× n holds for every k ∈ℕ.


The infinite horizon BRL relies on the stabilizing solution K^γ
={K^γ(ℓ)}_ℓ∈Θ∈𝕊ℍ_∞^n satisfying the following Θ-coupled ARE:

    K^γ(ℓ)=Ψ_1(K^γ)(ℓ)
    -Ψ_2(K^γ)(ℓ)
    Ψ_3^γ(K^γ)(ℓ)^-1Ψ_2(K^γ)(ℓ)'.


We call that K^γ∈𝕊ℍ_∞^n is a stabilizing solution of ARE (<ref>)
if ℱ(K^γ)
∈ℍ_∞^r× n
exponentially mean-square stabilizes Φ_v with conditioning,
i.e., the closed-loop system x(k+1)=[A(ϑ(k))+B(ϑ(k)) ·ℱ(K^γ)(ϑ(k))]x(k)
with the initial conditions (x_0,ϑ_0) is EMSS-C.


We are now in a position to state the infinite horizon BRL.


Given γ>0, Φ_v is internally stable and satisfies 𝔏_∞<γ
iff (<ref>) admits a stabilizing solution K^γ∈ℍ_∞^n+ meeting the sign condition
Ψ_3^γ(K^γ)(ℓ)≤-η_0 I_r× r μ-a.e., ∀η_0∈(0,γ^2-𝔏_∞^2).


Sufficiency.
If (<ref>) admits a stabilizing solution K^γ∈ℍ_∞^n+, then K̂^γ=-K^γ exists
as a stabilizing solution of

    K̂^γ(ℓ)=Ψ̂_1(K̂^γ)(ℓ)
    -Ψ_2(K̂^γ) Ψ̂_3(K̂^γ)(ℓ)^-1Ψ_2(K̂^γ)(ℓ)'

and satisfies
Ψ̂_3(K̂^γ)(ℓ)≥η_0 I_r× r μ-a.e., ∀η_0∈(0,γ^2
-𝔏_∞^2),
where Ψ̂_1(·)(ℓ):=𝒯_A(·)(ℓ)-C(ℓ)'C(ℓ)
and Ψ̂_3(·)(ℓ):=𝒯_B(·)(ℓ)-D(ℓ)'D(ℓ)+γ^2 I_r× r.
Following a similar argument as the proof of Theorem 13 in <cit.>,
one can yield that there are ρ>0, Y ∈𝕊ℍ_∞^n such that

    [[ Ψ̂_1(Y)(ℓ)-Y(ℓ)       Ψ_2(Y)(ℓ);      Ψ_2(Y)'(ℓ)      Ψ̂_3(Y)(ℓ) ]] ≥ρ I_(n+r)× (n+r), ℓ∈Θ.

And then, Y(ℓ) ≤K̂^γ(ℓ) ≤ 0, ℓ∈Θ can be shown by a similar proof with Theorem 11 in <cit.>.
Moreover, Ψ̂_1(Y)(ℓ)-Y(ℓ) ≥ρ I_n× n,
which leads to -Y(ℓ)-𝒯_A(-Y)(ℓ) ≥ρ I_n× n with -Y(ℓ) ≥ρ I_n× n.
Hence, by (iv) of Theorem <ref>, Φ_v is internally stable.
Next, we will prove 𝔏_∞<γ.
Consider Φ_v with x_0=0 and let T∈ℕ be arbitrary but fixed.
By completing the square associated with (<ref>),
(<ref>) can be rewritten as
J_γ(T,0,ℓ,v)=-𝔼{ϕ_x(T+1,0,ℓ,v)' K^γ(ϑ(T+1))
ϕ_x(T+1,0,ℓ,v)|ϑ_0=ℓ}
+∑_k=0^T𝔼{[v(k)-v̂(k)]'Ψ_3^γ(K^γ)(ϑ(k))
·[v(k)-v̂(k)]|ϑ_0=ℓ}≤ 0,∀ v∈ l^2_Θ(0,T;ℝ^r),  ℓ∈Θ, where v̂(k)=ℱ(K^γ)(ϑ(k))ϕ_x(k,0,ℓ,v).
Taking T →∞ in J_γ(T,0,ℓ,v) and applying Corollary <ref>,
we have that J_γ(∞,0,ℓ,v)=0 iff v(k)=v̂(k),
which contradicts with v≠0.
So, J_γ(∞,0,ℓ,v)≤-η_0v-v̂^2
_l^2_ℓ(ℕ;ℝ^r)<0 μ-a.e.,
and 𝔏_∞<γ follows from Proposition <ref>.

Necessity.
In view of Lemma <ref>, if Φ_v is internally stable and 𝔏_∞<γ,
then (<ref>) admits a solution K^γ(k)∈ℍ_∞^n+
satisfying Ψ_3^γ(K^γ(k))(ℓ)≤-η_0 I_r× r μ-a.e., ∀η_0∈(0,γ^2-𝔏_∞^2), and there exists ζ>0 such that
0≤ K^γ(k,ℓ)≤ K^γ(k+1,ℓ)≤ζ I_n× n μ-a.e., ∀ k∈ℕ.
Therefore, by the standard monotonicity result for the bounded positive semidefinite matrices,
there exists K^γ={K^γ(ℓ)}_ℓ∈Θ∈ℍ_∞^n+ satisfying lim_k→∞K^γ(k,ℓ)=K^γ(ℓ)≤ζ I_n× n.
According to the dominated convergence theorem,
lim_k→∞ℰ(K^γ(k))(ℓ)=∫_Θg(s|ℓ)
lim_k→∞K^γ(k,s)μ(ds) =ℰ(K^γ)(ℓ).
This concludes that K^γ solves (<ref>) after taking the limit in (<ref>) as k→∞.
Moreover,
Ψ_3^γ(K^γ)(ℓ)≤-η_0 I_r× r μ-a.e.,  ∀η_0∈(0,γ^2-𝔏_∞^2).
Next, we will prove that K^γ is a stabilizing solution of (<ref>).
For any v ∈ l^2_Θ(ℕ;ℝ^r) and τ>0,
introduce the perturbation operator
𝔏_∞^τ:l^2_Θ(ℕ;ℝ^r)→ l^2_Θ(ℕ;ℝ^m)
as 𝔏_∞^τ(v)(k,ℓ)=C_τ(ϑ(k))x(k,0,ℓ,v)+D_τ(ϑ(k))v(k),
where C_τ(·)=( C(·)',τ  I_n× n)' and
                   D_τ(·)=(D(·)',                   0 )'.
In consideration of 𝔏_∞<γ,
then for a sufficiently small τ>0,
𝔏_∞^τ<γ.
By Lemma <ref>, there exists K^γ,τ(k)∈ℍ_∞^n+, k∈ℕ satisfying

    {[               K^γ,τ(k+1,ℓ)=Ψ_1(K^γ,τ(k))(ℓ)
    + τ^2  I_n× n; -Ψ_2(K^γ,τ(k))(ℓ)
    Ψ_3^γ(K^γ,τ(k))(ℓ)^-1Ψ_2(K^γ,τ(k))(ℓ)',;      Ψ_3^γ(K^γ,τ(k))(ℓ)≤-η_0 I_r× r, ∀η_0∈(0,γ^2-𝔏^τ_∞^2) ].

with the initial condition K^γ,τ(0,ℓ)=0, ℓ∈Θ.
And for some ζ_τ>0,
0≤ K^γ(k,ℓ)≤ K^γ,τ(k,ℓ)≤ K^γ,τ(k+1,ℓ)≤ζ_τ I_n× n
holds for every k∈ℕ, ℓ∈Θ,
where K^γ(k) is the solution of (<ref>).
Therefore, lim_k→∞K^γ,τ(k)
=K^γ,τ∈ℍ_∞^n+ exists
and solves the following ARE:

    K^γ,τ(ℓ)=   -Ψ_2(K^γ,τ)(ℓ)
    Ψ_3^γ(K^γ,τ)(ℓ)^-1
    ·Ψ_2   (K^γ,τ)(ℓ)'
    +Ψ_1(K^γ,τ)(ℓ)+τ^2  I_n× n

with Ψ_3^γ(K^γ,τ)(ℓ)≤-η_0 I_r× r
for any η_0∈(0,γ^2-𝔏^τ_∞^2).
Moreover, K^γ≤ K^γ,τ.
Subtracting (<ref>) from (<ref>), it follows that
K^γ,τ(ℓ)-K^γ(ℓ)
=[A(ℓ)+B(ℓ)ℱ(K^γ)(ℓ)]'ℰ(K^γ,τ-K^γ)(ℓ)
[A(ℓ)+B(ℓ)ℱ(K^γ)(ℓ)]+M(ℓ),
where M(ℓ)=τ^2 I_n× n-[ℱ(K^γ)(ℓ)-ℱ(K^γ,τ)(ℓ)]'
·Ψ_3^γ(K^γ,τ)(ℓ)[ℱ(K^γ)(ℓ)-ℱ(K^γ,τ)(ℓ)] ≥τ^2 I_n× n.
This leads to K^γ,τ(ℓ)-K^γ(ℓ)≥τ^2 I_n× n.
By Theorem <ref>,
F(K^γ) exponentially stabilizes Φ_v,
i.e., K^γ is a stabilizing solution of (<ref>).


For finite MJLSs, with the intention of deriving the infinite horizon BRL,
it is typical in previous literature to assume the weak controllability of the system (see, <cit.>)
or that the transition probability matrix of the Markov chain is nondegenerate (see, <cit.>).
Note that the latter is less restrictive than the former.
In this paper, we adopt Assumption <ref>,
which corresponds to the transition probability matrix being nondegenerate in the finite scenario.


In what follows, a backward iteration will be presented for solving (<ref>),
whenever (<ref>) admits a stabilizing solution satisfying the sign condition in Theorem <ref>.
To this end, for every T∈ℕ, we consider the backward difference equation (<ref>).
Applying Theorem <ref>, we confirm that (𝐀;𝒢) is EMSS-C and Φ_v satisfies 𝔏_∞<γ.
Following a similar argument as the proof of Lemma <ref>,
it can be concluded that (<ref>) admits a solution Y_T^γ(k), k∈0, T satisfying the sign condition Ψ_3^γ(Y_T^γ(k))(ℓ)≤-η_0 I_r× r, ℓ∈Θ.
Moreover, 0≤ Y_T^γ(k,ℓ)≤ Y_T+1^γ(k,ℓ)≤ζ I_n× n μ-a.e.,
where Y_T+1^γ(k), k∈0,T+1 is the solution of (<ref>) with Y(T+2,ℓ)=0 and ζ>0.
Therefore, {Y_T^γ(0)}_T∈ℕ is a monotone increasing sequence which is bounded above.
It then follows from the standard monotonicity result concerning the bounded positive semidefinite matrices
that lim_T→+∞Y_T^γ(0)=lim_T→+∞Y_T+1^γ(0)=Y^γ.
Take into account the difference equation  Y_T+1^γ(0)=Ψ_1(Y_T+1^γ(1))
-Ψ_2(Y_T+1^γ(1))Ψ_3^γ(Y_T+1^γ(1))^-1Ψ_2(Y_T+1^γ(1))'.
Since Y^γ_T+1(1)=Y^γ_T(0), ∀ T∈ℕ,
taking T→∞,
we know that Y^γ satisfies (<ref>).
From what has been discussed above,
we propose a backward iteration summarized as Algorithm <ref>
for getting a numerical solution of a positive semidefinite solution to (<ref>).


It should be noted that Algorithm <ref> is efficient to implement,
but it has a limitation since it only solves the numerical solution of a positive semidefinite solution
to (<ref>) rather than the stabilizing solution.
Regarding to finite MJLSs,
one can refer to <cit.> for an iterative deterministic algorithm of the stabilizing solutions to a class of stochastic Riccati equations,
in which the original problem is transformed into solving a sequence of uncoupled deterministic Riccati equations.
However, for the ARE (<ref>) coupled via the integral, it is significantly challenging to obtain the auxiliary uncoupled Riccati equations. Additionally, we recognize that the existence conditions for the stabilizing solution of the ARE is an issue of considerable importance.
<cit.> has made a great progress for finite MJLSs.
In future research, we hope to do some in-depth studies on the conditions for the existence of the stabilizing solution of the ARE
and the numerical algorithm for the case where the Markov chain takes values in Θ.
We thank the anonymous reviewer for providing valuable research ideas for us.




§ EXAMPLES


In this section, several examples are supplied to illustrate the feasibility of our work.

The results of analysis and synthesis of discrete-time MJLSs,
where the Markov chain takes values in Θ,
potentially provide significant value for some models constructed based on actual applications (see <cit.>).
For example, it has been applied to networked control systems with the continuous-valued random delays (see, <cit.>).
In this example, we consider the solar thermal receiver model proposed in <cit.>,
which was also discussed in Chapter 8 of <cit.>.
In these works, the influence of atmospheric conditions on the system dynamics is usually modeled by a two-mode Markov chain,
representing two atmospheric conditions: 1) sunny and 2) cloudy.
However, even under the sunny or cloudy atmospheric condition,
the instantaneous solar radiation still fluctuates and then heavily influences the system dynamics.
Therefore, it is reasonable to establish a more accurate model by using a Markov chain that takes values in Θ
to describe the effect of varying solar radiation intensities on the system dynamics.
In <cit.>, the system dynamics is modeled by x(k+1)=a(ϑ(k))x(k), k∈ℕ and the Markov chain {ϑ(k)}_k∈ℕ is on the Borel space (Θ, ℬ(Θ)),
where Θ=Θ_1⋃Θ_2
with Θ_i={i× [0,1]}, i∈{1,2}.
For ϑ(k)=(i,t) with t∈[0,1], i=1 (i=2) represents the sunny (cloudy) atmospheric condition,
while t represents the effects of instantaneous insolation on the system parameters under the sunny (cloudy) atmospheric condition.
Define the measure μ on ℬ(Θ) as μ({i}× M)=μ_L(M),
where μ_L(·) is the Lebesgue measure, M is a Borel set on [0,1] and i∈{1,2}.
When ϑ(k)=(i,t) and ϑ(k+1)=(j,S) with S being uniformly distributed over the interval [0,1],
let j=1 with probability P_i1 and j=2 with  probability P_i2 be the transition probability function.
Consider the parameters given in <cit.>: a_11=0.9, a_12=0.7, a_21=0.95, a_22=1.15.
And for a={a(ℓ)}_ℓ∈Θ, when ℓ=(i,t),
set a(ℓ)=a_i1+t(a_i2-a_i1), i∈{1,2}, t∈[0,1].
Here, take P_11=0.9767, P_22=0.7611.
Since U(ℓ)=[a(ℓ)]^2 and ξ=0.0047 solves U(ℓ)-𝒯_a(U)(ℓ)≥ξ μ-a.e.,
from (iv) of Theorem <ref>, one can infer that the system is EMSS-C.


First, we consider a two-dimension MJLS (𝐀,P): x(k+1)=A(ϑ(k))x(k), k∈ℕ
and the Markov chain {ϑ(k)}_k∈ℕ takes values in a finite set {1,2}.
Define the measure on {1,2} as the counting measure,
and give the probability transition matrix of the Markov chain by P=(p_ij)_2× 2 with p_11=0.15 and p_22=0.1.
The system coefficients are taken as A(ϑ(k))=A_11 for ϑ(k)=1 and A(ϑ(k))=A_21 for ϑ(k)=2,
where
A_11=
(
   [  2 -1;  0  0;    ]),
A_21=
(
   [ 0 1; 0 2;   ]).
It is not difficult to know that there are instable individual modes for the finite MJLS (𝐀,P).
However, we can find that Y={Y(i)}_i∈1,2 with Y(i)=Y_i∈𝕊^2+* solves the Lyapunov-type inequality:
-Y(i)+A(i)'[∑_j=1^2 p_ijY(j)]A(i)+I_2× 2≤ 0,  ∀ i∈1,2,
where
Y_1=10^3×(
                [  1.3438 -0.6177; -0.6177  0.4501;         ]),
Y_2=10^3×(
                [  0.1104 -0.0044; -0.0044  1.3873;         ]).
According to (iv) of Theorem <ref>, it follows that (𝐀,P) is EMSS-C,
thereby (𝐀,P) is EMSS by Proposition <ref>.
Or directly from Theorem <ref>,
the EMSSy of (𝐀,P) is immediately available since r_σ(ℒ_A)=0.6<1.
Therefore, the instability of individual modes does not imply that the finite MJLS is not EMSS
(one can refer to <cit.> for other examples).

Next, we consider a MJLS with the Markov chain on a Borel space,
which can be viewed as the uncountably infinite scenario corresponding to (𝐀;P) discussed above.
Consider the  MJLS (𝐀;𝒢) with {ϑ(k)}_ k∈ℕ being on (Θ, ℬ(Θ)),
where Θ and the measure μ are the same as those in Example <ref>.
The transition probability function is given as follows:
𝒫{ϑ(k+1)∈Θ_j|ϑ(k)∈Θ_i}=p_ij, i∈1,2, j∈1,2.
As for A={A(ℓ)}_ℓ∈Θ of (𝐀;𝒢),
let A(ℓ)=A(1,t)=A_11+t(A_12-A_11) for ℓ=(1,t)∈Θ_1
and A(ℓ)=A(2,t)=A_21+t(A_22-A_21) for ℓ=(2,t)∈Θ_2, 0≤ t≤ 1,
where A_12=1/2A_11, A_22=1/2A_21.
Although there are instable individual modes,
we find that Y={Y(ℓ)}_ℓ∈Θ∈ℍ_∞^2+* with Y(ℓ)=Y(i,t) for ℓ=(i,t)
solves -Y(i,t)+A(i,t)'[∑_j=1^2∫^1_0 p_ijY(j,s)ds]A(i,t)+I_2× 2≤ 0, ∀ i∈1,2, t∈[0,1],
where Y(1,t)=Y_1 and Y(2,t)=Y_2.
From (iv) of Theorem <ref>, (𝐀;𝒢) is EMSS-C.
Furthermore, by Proposition <ref>, we deduce that (𝐀;𝒢) is EMSS.
Hence, for the case where the Markov chain takes values in Θ,
the instability of individual modes does not imply that the MJLS is not EMSS,
which is consistent with the finite case.


In this example, consider a two-dimension MJLS Φ_v,
where {ϑ(k)}_k∈ℕ and A are the same as those of (𝐀;𝒢) in Example <ref>.
It has been shown in Example <ref> that (𝐀;𝒢) is EMSS-C.
Set B(ℓ)=tB_i, C(ℓ)=tC_i, D(ℓ)=0 for ℓ=(i,t)∈Θ_i, i∈1,2, t∈[0,1], where
B_1=([  0.4 -0.2 ])',
C_1=([ -0.12  -0.3 ]),
B_2=([ -1.1  0.3 ])',
C_2=([ 0.4 0.2 ]).
And take γ=0.5.
Applying Algorithm <ref> with the computational accuracy ϵ=10^-5
and considering the finite grid approximation for Θ,
we get a numerical solution K={K(ℓ)}_ℓ∈Θ of (<ref>) presented in Fig.<ref>,
in which K(ℓ)=K(i,t) is a positive semidefinite matrix for any ℓ=(i,t)∈Θ_i, i∈{1,2}, t∈[0,1].
Moreover, K satisfies B(i,t)'[∑_j=1^2∫^1_0 p_ijK(j,s)ds]B(i,t)-γ^2<0.
Substituting K into F=ℱ(K),
it can also be verified that Y={Y(ℓ)}_ℓ∈Θ
with Y(ℓ)=Y_i∈𝕊^2+* for ℓ=(i,t)∈Θ_i as given in Example <ref>,
satisfies
-Y(i,t)+[A(i,t)+B(i,t)F(i,t)]'[∑_j=1^2∫^1_0 p_ijY(j,s)ds] ·[A(i,t)+B(i,t)F(i,t)]+I_2× 2≤ 0,
i∈{1,2}, t∈[0,1].
These ensure that K is the stabilizing solution of (<ref>) and satisfies the required sign condition.

Now, let us take an exogenous disturbance signal v={v(k)}_k∈ℕ with finite energy, where v(k)=e^-2k.
Fig. <ref> demonstrates 100 possible trajectories of Φ_v,
and the mean value of the system state is marked.
During this simulation, the sampling period d is 0.01 hours.
From Fig. <ref>, one can see that for any sample time s∈{d, 2d, 3d,⋯, 400d},
y_l^2(0,s;R)/v_l^2(0,s;R)≤γ,
which verifies the feasibility of Theorem <ref>.







§ CONCLUSION


This paper has studied exponential stability and the disturbance attenuation property
for discrete-time MJLSs with the Markov chain on a Borel space (Θ, ℬ(Θ)).
The results developed could be viewed as an extension of the previous ones
for MJLSs with the Markov chain on a countable set to the uncountable scenario.
First, two kinds of exponential stabilities have been introduced: EMSSy and EMSSy-C.
The spectral criteria have been proposed for EMSSy and EMSSy-C respectively,
and the Lyapunov-type criteria have also been established for EMSSy-C.
As for the relationship between these two kinds of exponential stabilities,
we are of the opinion that EMSSy-C and EMSSy are equivalent (see Conjecture <ref>),
and some discussions have been made in Proposition <ref> and Remark <ref>.
Regarding the disturbance attenuation property,
the BRLs have been presented in terms of the Θ-coupled DRE for the finite horizon case
and ARE for the infinite horizon case.
As the core of H_∞ analysis, the BRL will play a significant role in more studies on the system synthesis issues,
such as H_∞ filter and H_∞ controller designs for MJLSs.
In addition, it is worthwhile to note that the results drawn in this paper on exponential stability and BRLs
can be directly extended to the system with multiple noises.


42
natexlab#1#1
[#1],#1[Aberkane(2013)]Aberkane2013
authorAberkane, S. (year2013).
titleBounded real lemma for nonhomogeneous Markovian
  jump linear systems.
journalIEEE Transactions on Automatic Control,
  volume58number(3), pages797–801.
[Aberkane and Dragan(2020)]2020AberkaneJDEA
authorAberkane, S., & authorDragan, V.
  (year2020).
titleOn the existence of the stabilizing solution of
  generalized Riccati equations arising in zero-sum stochastic difference
  games: The time-varying case.
journalJournal of Difference Equations and
  Applications,  volume26, pages913–951.
[Aberkane and Dragan(2023)]2022Game
authorAberkane, S., & authorDragan, V.
  (year2023).
titleA deterministic setting for the numerical computation
  of the stabilizing solutions of stochastic game-theoretic Riccati
  equations.
journalMathematics,  volume11, pages2068.
[Billingsley(1995)]BookBillingsley1995
authorBillingsley, P. (year1995).
titleProbability and Measure (Third Edition).
addressNew York: publisherJohn Wiley &
  Sons, Inc.

[Cohn(2013)]Cohn2013
authorCohn, D. L. (year2013).
titleMeasure Theory (Second Edition).
addressBasle: publisherBirkhäuser.

[Costa and Figueiredo(2014)]Costa2014
authorCosta, O. L. V., & authorFigueiredo, D. Z.
  (year2014).
titleStochastic stability of jump discrete-time linear
  systems with Markov chain in a general Borel space.
journalIEEE Transactions on Automatic Control,
  volume59number(1), pages223–227.
[Costa and Figueiredo(2015)]Costa2015
authorCosta, O. L. V., & authorFigueiredo, D. Z.
  (year2015).
titleLQ control of discrete-time jump systems with
  Markov chain in a general Borel space.
journalIEEE Transactions on Automatic Control,
  volume60number(9), pages2530–2535.
[Costa and Figueiredo(2016)]Costa2016
authorCosta, O. L. V., & authorFigueiredo, D. Z.
  (year2016).
titleQuadratic control with partial information for
  discrete-time jump systems with Markov chain in a general Borel space.
journalAutomatica,  volume66,
  pages73–84.
[Costa and Figueiredo(2017)]Costa2017
authorCosta, O. L. V., & authorFigueiredo, D. Z.
  (year2017).
titleFiltering 𝒮-coupled algebraic Riccati
  equations for discrete-time Markov jump systems.
journalAutomatica,  volume83,
  pages47–57.
[Costa and Fragoso(1995)]Costa1995
authorCosta, O. L. V., & authorFragoso, M. D.
  (year1995).
titleDiscrete-time LQ-optimal control problems for
  infinite Markov jump parameter systems.
journalIEEE Transactions on Automatic Control,
  volume40number(12), pages2076–2088.
[Costa et al.(2005)Costa, Fragoso and Marques]BookCosta2005
authorCosta, O. L. V., authorFragoso, M. D., &
  authorMarques, R. P. (year2005).
titleDiscrete-Time Markov Jump Linear Systems.
addressLondon: publisherSpringer-Verlag.
[Dragan et al.(2010)Dragan, Morozan and Stoica]BookDragan2010
authorDragan, V., authorMorozan, T., &
  authorStoica, A.-M. (year2010).
titleMathematical Methods in Robust Control of
  Discrete-Time Linear Stochastic Systems.
addressNew York: publisherSpringer-Verlag.
[Dragan et al.(2014)Dragan, Morozan and Stoica]BookDragan2014
authorDragan, V., authorMorozan, T., &
  authorStoica, A.-M. (year2013).
titleMathematical Methods in Robust Control of Linear Stochastic Systems (Second Edition).
addressNew York: publisherSpringer-Verlag.
[Fang and Loparo(2002)]Fangyg2002
authorFang, Y., & authorLoparo, K. A.
  (year2002).
titleStochastic stability of jump linear systems.
journalIEEE Transactions on Automatic Control,
  volume47number(7), pages1204–1208.
[Feng et al.(1992)Feng, Loparo, Ji and Chizeck]Fengxial1992
authorFeng, X., authorLoparo, K. A.,
  authorJi, Y., & authorChizeck, H. J.
  (year1992).
titleStochastic stability properties of jump linear
  systems.
journalIEEE Transactions on Automatic Control,
  volume37number(1), pages38–53.
[Fragoso and Baczynski(2002)]Fragosojack2002
authorFragoso, M. D., & authorBaczynski, J.
  (year2002).
titleStochastic versus mean square stability in continuous
  time linear infinite Markov jump parameter systems.
journalStochastic Analysis and Applications,
  volume20number(2), pages347–356.
[Gao et al.(2023)Gao, Deng, Zhuang and Zeng]SCIchinadeng2023
authorGao, X., authorDeng, F., authorZhuang,
  H., & authorZeng, P. (year2023).
titleObserver-based event-triggered asynchronous control
  of networked Markovian jump systems under deception attacks.
journalScience China-Information Sciences,  volume66, pages159204.
[Hou et al.(2021)Hou, Liu and Deng]SCIchinaHou2021
authorHou, T., authorLiu, Y., & authorDeng,
  F. (year2021).
titleStability for discrete-time uncertain systems with
  infinite Markov jump and time-delay.
journalScience China-Information Sciences,  volume64, pages152202.
[Hou and Ma(2016)]Hou2016AC
authorHou, T., & authorMa, H. (year2016).
titleExponential stability for discrete-time infinite
  Markov jump systems.
journalIEEE Transactions on Automatic Control,
  volume61number(12), pages4241–4246.
[Hou et al.(2016)Hou, Ma and Zhang]Hout2016auto
authorHou, T., authorMa, H., & authorZhang,
  W. (year2016).
titleSpectral tests for observability and detectability of
  periodic Markov jump systems with nonhomogeneous Markov chain.
journalAutomatica,  volume63,
  pages175–181.
[Jutinico et al.(2021)Jutinico, Flórez-cediel and Siqueira]Andres2021
authorJutinico, A. L., authorFlórez-cediel, O., &
  authorSiqueira, A. A. (year2021).
titleComplementary stability of Markovian systems:
  Series elastic actuators and human-robot interaction.
journalIFAC-PapersOnLine,  volume54number(4), pages112–117.
[Kallenberg(2002)]Kallenberg2002
authorKallenberg, O. (year2002).
titleFoundations of Modern Probability (Second Edition).
addressNew York: publisherSpringer-Verlag.
[Kechris(1995)]Kechris1995
authorKechris, A. S. (year1995).
titleClassical Descriptive Set Theory.
addressNew York: publisherSpringer-Verlag.
[Kordonis and Papavassilopoulos(2014)]Ioannis2014
authorKordonis, I., & authorPapavassilopoulos, G. P.
  (year2014).
titleOn stability and LQ control of MJLS with a
  Markov chain with general state space.
journalIEEE Transactions on Automatic Control,
  volume59number(2), pages535–540.
[Li et al.(2012)Li, Chen, Lam and Mao]Li2012
authorLi, C., authorChen, M. Z. Q.,
  authorLam, J., & authorMao, X.
  (year2012).
titleOn exponential almost sure stability of random jump
  systems.
journalIEEE Transactions on Automatic Control,
  volume57number(12), pages3064–3077.
[Lin et al.(2016)Lin, Liu, Zhang and Niu]Lin2016
authorLin, Z.,
authorLiu, J., authorZhang, W., & authorNiu, Y.
  (year2016).
titleRegional pole placement of wind turbine generator
  system via a Markovian approach.
journalIET Control Theory & Applications,
  volume10number(15), pages1771–1781.
[Liu et al.(2009)Liu, Ho and Niu]Liuming2009
authorLiu, M., authorHo, D. W., & authorNiu, Y.
  (year2009).
titleStabilization of Markovian jump linear system over
  networks with random communication delay.
journalAutomatica,  volume45number(2),
  pages416–421.
[Lutz and Stilwell(2016)]Collin2016
authorLutz, C. C., & authorStilwell, D. J.
  (year2016).
titleStability and disturbance attenuation for Markov
  jump linear systems with time-varying transition probabilities.
journalIEEE Transactions on Automatic Control,
  volume61number(5), pages1413–1418.
[Meyn and Tweenie(2009)]Meynsp2009
authorMeyn, S., & authorTweenie, R. L.
  (year2009).
titleMarkov Chain and Stochatsic Stability (Second Edition).
addressNew York: publisherCambridge University
  Press.
[Seiler and Sengupta(2003)]Seiler2003
authorSeiler, P., & authorSengupta, R.
  (year2003).
titleA bounded real lemma for jump systems.
journalIEEE Transactions on Automatic Control,
  volume48number(9), pages1651–1654.
[Shi et al.(2022)Shi, Shi and Wu]ShiT2022
authorShi, T., authorShi, P., & authorWu,
  Z. (year2022).
titleDynamic event-triggered asynchronous MPC of
  Markovian jump systems with disturbances.
journalIEEE Transactions on Cybernetics,  volume52number(11), pages11639–11648.
[Si et al.(2020)Si, Ni and Zhang]SiB2020
authorSi, B., authorNi, Y., & authorZhang,
  J. (year2020).
titleTime-inconsistent stochastic LQ problem with regime
  switching.
journalJournal of Systems Science & Complexity,
   volume33, pages1733–1754.
[Sworder and Rogers(1983)]Sworder1983
authorSworder, D., & authorRogers, R.
  (year1983).
titleAn LQ-solution to a control problem associated with
  a solar thermal central receiver.
journalIEEE Transactions on Automatic Control,
  volume28number(10), pages971–978.
[Todorov and Fragoso(2011)]Marcosgtmf2011
authorTodorov, M. G., & authorFragoso, M. D.
  (year2011).
titleOn the robust stability, stabilization, and stability
  radii of continuous-time infinite Markov jump linear systems.
journalSIAM Journal on Control and Optimization,
  volume49number(3), pages1171–1196.
[Todorov et al.(2018)Todorov, Fragoso and Costa]Marcos2018auto
authorTodorov, M. G., authorFragoso, M. D., &
  authorCosta, O. L. V. (year2018).
titleDetector-based H_∞ results for
  discrete-time Markov jump linear systems with partial observations.
journalAutomatica,  volume91,
  pages159–172.
[Ungureanu(2014)]Vitoricam2014
authorUngureanu, V. M. (year2014).
titleStability, stabilizability and detectability for
  Markov jump discrete-time linear systems with multiplicative noise in
  Hilbert spaces.
journalOptimization,  volume63number(11), pages1689–1712.
[Ungureanu et al.(2012)Ungureanu, Dragan and
  Morozan]Ungureanu2013opcam
authorUngureanu, V. M., authorDragan, V., &
  authorMorozan, T. (year2013).
titleGlobal solutions of a class of discrete-time backward
  nonlinear equations on ordered Banach spaces with applications to Riccati
  equations of stochastic control.
journalOptimal Control Applications and Methods,
  volume34number(2), pages164–190.
[Wakaiki et al.(2018)Wakaiki, Ogura and Hespanha]Masashi2018
authorWakaiki, M., authorOgura, M., &
  authorHespanha, J. P. (year2018).
titleLQ-optimal sampled-data control under stochastic
  delays: Gridding approach for stabilizability and detectability.
journalSIAM Journal on Control and Optimization,
  volume56number(4), pages2634–2661.
[Wonham(1968)]Wonham1968
authorWonham, W. M. (year1968).
titleOn a matrix Riccati equation of stochastic
  control.
journalSIAM Journal on Control and Optimization,
  volume6number(4), pages681–697.
[Xue et al.(2021)Xue, Yan, Zhang, Shen and Shi]XueM2021
authorXue, M., authorYan, H., authorZhang,
  H., authorShen, H., & authorShi, P.
  (year2021).
titleDissipativity-based filter design for Markov jump
  systems with packet loss compensation.
journalAutomatica,  volume133, pages109843.
[Zhang(2009)]Zhanglx2009
authorZhang, L. (year2009).
titleH_∞ estimation for discrete-time piecewise
  homogeneous Markov jump linear systems.
journalAutomatica,  volume45number(11),
  pages2570–2576.
[Zhou and Luo(2018)]Zhoubin2018
authorZhou, B., & authorLuo, W.
  (year2018).
titleImproved Razumikhin and Krasovskii stability
  criteria for time-varying stochastic time-delay systems.
journalAutomatica,  volume89,
  pages382–391.



