Stability of Markov regenerative switched linear systems ?
Masaki Ogura a , Victor M. Preciado a

arXiv:1501.03474v2 [cs.SY] 5 Dec 2015

a

Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA 19104-6314, USA

Abstract
In this paper, we give a necessary and sufficient condition for mean stability of switched linear systems having a Markov
regenerative process as its switching signal. This class of switched linear systems, which we call Markov regenerative switched
linear systems, contains Markov jump linear systems and semi-Markov jump linear systems as special cases. We show that a
Markov regenerative switched linear system is mth mean stable if and only if a particular matrix is Schur stable, under the
assumption that either m is even or the system is positive.
Key words: Switched linear systems; mean stability; Markov regenerative processes; positive systems

1

Introduction

Among switched linear systems, those having a timehomogeneous Markov process as its switching signal,
called Markov jump linear systems (Costa, Fragoso &
Todorov 2013), are of particular importance. One of the
reasons for their importance is that time-homogeneous
Markov processes are well suited to model stochastic
phenomena presenting a constant rate of occurrence. Another reason is that the analysis and synthesis of Markov
jump linear systems can be performed in a rather similar way to those for linear time-invariant systems by the
introduction of auxiliary variables (Costa et al. 2013).

processes that can be obtained by concatenating independent and identically distributed random functions
(see Sigman & Wolff (1993) for the details). The mean
stability analysis of linear systems subject to regenerative switchings is performed in Ogura & Martin (2015).

However, it is often restricting to assume that the switching signals present constant transition rates, or even
the Markovian property. To overcome this restriction,
we find a wide variety of alternative switching signals
in the literature (Hou, Luo & Shi 2005, Antunes, Hespanha & Silvestre 2013, Ogura & Martin 2015, Ogura &
Martin 2014). A natural extension to time-homogeneous
Markov processes are time-homogeneous semi-Markov
processes, which are Markovian-like processes with timevarying transition rates (see Çinlar 1975). The stability
analysis of the corresponding switched linear systems
can be found in Antunes et al. (2013) and Ogura & Martin (2014). Another extension are regenerative processes
(Smith 1955), which are, roughly speaking, stochastic

In this paper, we extend the works in Antunes et al.
(2013), Ogura & Martin (2015), and Ogura & Martin (2014) to analyze the stability of Markov regenerative switched linear systems, which are switched linear systems whose switching signal is a Markov regenerative process (also called a semi-regenerative process)
(Çinlar 1975, Choi, Kulkarni & Trivedi 1994). Markov
regenerative processes form a large class of stochastic
processes which contains as special cases all the Markov,
semi-Markov, and regenerative processes. We show that
exponential mth mean stability of a Markov regenerative
switched linear system is characterized by the spectral
radius of a matrix, under the assumption that either m is
even or the system is positive. Extending various results
in the literature (Antunes et al. 2013, Ogura & Martin
2015, Ogura & Martin 2014, Fang & Loparo 2002), the
obtained result enables us to analyze the stability of, for
example, state-feedbacked Markov jump linear systems
with periodically observed mode signals (see a discretetime setting in Cetinkaya & Hayakawa (2015)), as well
as controlled system with failure-prone controllers with
only one repairing facility (Distefano & Trivedi 2013,
Section 3.2).

? This paper was not presented at any IFAC meeting. Corresponding author M. Ogura.
Email addresses: ogura@seas.upenn.edu (Masaki
Ogura), preciado@seas.upenn.edu (Victor M. Preciado).

The paper is organized as follows. After introducing necessary notations, in Section 2 we introduce the class of
Markov regenerative switched linear systems under consideration and then state a necessary and sufficient con-

Preprint submitted to Automatica

8 December 2015

We call τ the regeneration times of σ, and (θ, τ ) the
embedded Markov renewal process of σ.

dition for their exponential mean stability. Then, in Section 3, we present various applications of the main result. The notation used in this paper is standard. When
x ∈ Rn is nonnegative entrywise we write x ≥ 0. The
standard Euclidean norm on Rn is denoted by k·k. Let
I and O denote the identity and zero matrices, respectively. The block diagonal matrix with block diagonals
LN
A1 , . . . , AN is denoted by i=1 Ai . The kronecker product of two matrices A and B is denoted by A ⊗ B. We
say that A ∈ Rn×n is Schur stable if A has the spectral
radius less than one. Also we say that A is Hurwitz stable if all the eigenvalues of A have negative real parts.
For an integrable random variable X, its expected value
is denoted by E[X] and its conditional expectations by
E[X | ·].
2

Among the three conditions in the definition, (P3) is the
most important. It implies that, as far as prediction is
concerned, at the regeneration time τk , the past information of the process {σs }s≤τk is irrelevant and only the
value of θk is needed. Actually, (P3) implies that
P (στk +t = λ | θk = j, σs ; s ≤ τk ) = P (σt = λ | θ0 = j)
(2)
for all λ ∈ Λ, i, j ∈ {1, . . . , N }, t ≥ 0, and k ≥ 0. Equation (2) indicates that, given θk = j, the process σ “regenerates” at time τk as if it starts from time 0, given
θ0 = j. Moreover, σ possesses a certain Markovian property at the regeneration time τk ; once we know the value
of στk , using (P2) we can determine the value θk , which
then determines the future distribution of σ by (2). As
is pointed out in Çinlar (1974), it is convenient to take π
to be the identity, but this is not necessary. Also we remark that (P1) is a technical condition needed to state
(P3). For the details, the readers are referred to Çinlar
(1975) and Çinlar (1974).

Stability characterization

This section introduces the class of Markov regenerative switched linear systems and then presents a necessary and sufficient condition for their stability. We
need to first recall the definition of Markov renewal processes (Çinlar 1975). Let N be a positive integer. A
stochastic process (θ, τ ) = {(θk , τk )}k≥0 taking values
in {1, . . . , N } × [0, ∞) and satisfying 0 = τ0 ≤ τ1 ≤ · · ·
is called a Markov renewal process (Çinlar 1975) if

Then, we introduce the class of switched linear systems
studied in this paper.
Definition 2 Let σ be a Markov regenerative process
and let Aλ ∈ Rn×n for each λ ∈ Λ. Then, the stochastic
differential equation

P (θk+1 = j, τk+1 − τk ≤ t | θk , . . . , θ0 , τk , . . . , τ0 )
= P (θk+1 = j, τk+1 − τk ≤ t | θk )
holds for every k and t ≥ 0. We assume that (θ, τ ) is
time-homogeneous, that is, for all i, j and t ≥ 0, the
probability P (θk+1 = j, τk+1 − τk ≤ t | θk = i) is independent of k. We note that, in this case, θ is a timehomogeneous Markov chain and therefore has the constant transition probabilities P (θk+1 = j | θk = i). In
this paper, we furthermore assume that
τk+1 − τk > 0

Σ:

dx
= Aσt x,
dt

(3)

where x(0) = x0 ∈ Rn is a constant, is called a Markov
regenerative switched linear system.
For example, a time-homogeneous Markov process r
is Markov regenerative with an underlying embedded
Markov renewal process being {(rtk , tk )}k≥0 , where
0 = t0 < t1 < t2 < · · · are the times at which the process r changes its value. Therefore, Markov jump linear
systems (Costa et al. 2013) are Markov regenerative
switched linear systems. In Section 3, we will show in detail that more general classes of switched linear systems,
such as semi-Markov jump linear systems (Antunes
et al. 2013, Ogura & Martin 2014) and regenerative
switched linear systems (Ogura & Martin 2015), are
contained in the class of Markov regenerative switched
linear systems.

(1)

with probability one for every k. Then, we can state the
definition of Markov regenerative processes as follows.
Definition 1 (Çinlar (1975), Choi et al. (1994))
Let σ = {σt }t≥0 be a stochastic process taking values in
a finite set Λ. We say that σ is Markov regenerative if
there exists a Markov renewal process (θ, τ ) taking values in {1, . . . , N } × [0, ∞) and satisfying the following
conditions for every k:
(P1) τk is a stopping time for σ;
(P2) There exists a function π : Λ → {1, . . . , N } such
that π(στk ) = θk ;
(P3) For j ∈ {1, . . . , N }, ` ≥ 1, 0 ≤ t1 < t2 < · · · < t` ,
and a function f : Λ` → [0, ∞),

Based on the embedded Markov renewal process, we
can naturally define the stability of Markov regenerative
switched linear systems as follows.
Definition 3 Given a positive integer m, we say that Σ
is exponentially mth mean stable if there exist C > 0
and β > 0 such that E[kx(t)km ] ≤ Ce−βt kx0 km for every
x0 and θ0 .

E[f (στk +t1 , . . . , στk +t` ) | θk = j, σs ; s ≤ τk ]
= E[f (σt1 , . . . , σt` ) | θ0 = j].

2

Also, we here introduce the positivity of Σ.

2.1

Definition 4 We say that Σ is positive if x0 ≥ 0 implies
x(t) ≥ 0 with probability one for all θ0 and t ≥ 0.

We present a proof of Theorem 5 in this subsection. This
proof consists of the following two steps. We first show,
in Proposition 6, that the stability of Σ can be analyzed
based on its discretized version. The proof of the proposition utilizes the technique developed in Ogura & Martin (2015) to study regenerative switched linear systems.
To analyze the discretized version of the system, we then
present Proposition 7, which extends Theorem 3.4 in
Ogura & Martin (2014) for not necessarily positive systems.

We remark that, for Σ to be positive, it is clearly sufficient that all the matrices Aλ (λ ∈ Λ) are Metzler,
i.e., the off-diagonal entries of each Aλ are all nonnegative (Farina & Rinaldi 2000). However, this sufficient
condition is not necessary as is shown in Ogura & Martin (2015, Example 10) for regenerative switched linear
systems.
In order to state the main result of this paper, we recall
the notion of induced matrices. First we define (see, e.g.,
Parrilo & Jadbabaie 2008) the m-lift of x ∈ Rn , denoted
by x[m] , as the real vector of length nm = n+m−1
m
with its elements
being the lexicographically ordered
√
monomials α! xα (α! := m!/(α1 ! · · · αn !)) that are indexed by all the possible exponents α = (α1 , . . . , αn )
summing up to m. For example, √
for x = [x1 , x2 ]>
[1]
[2]
we have x√
= x,
[x21 , 2x1 x2 , x22 ]> , and
√ x 2 =3 >
[3]
3
2
x = [x1 , 3x1 x2 , 3x1 x2 , x2 ] . Then, we define the
mth induced matrix of A, denoted by A[m] , as the
nm × nm unique matrix (Parrilo & Jadbabaie 2008)
satisfying (Ax)[m] = A[m] x[m] for every x ∈ Rn .

Proof

Let Σ be a Markov regenerative switched linear system. Then, the discretized process xd = {x(τk )}k≥0 is
the solution of the discrete-time system SΣ : xd (k +
1) = Φ(τk+1 ; τk )xd (k). In order to proceed, we introduce a class of switched linear systems called discretetime semi-Markov jump linear systems (Ogura & Martin
2014). Let {Fk }k≥0 be a stochastic process taking values in Rn×n . The system Σd : xd (k + 1) = Fk xd (k)
(xd (0) = x0 ) is said to be a discrete-time semi-Markov
jump linear system if there exists a time-homogeneous
Markov chain θ taking values in {1, . . . , N } such that,
for all k ≥ 0, i, j ∈ {1, . . . , N }, and a Borel subset G of
Rn×n , there holds that
P (θk+1 = j, Fk ∈ G | θk , . . . , θ0 , Fk−1 , . . . , F0 )
(5)
= P (θk+1 = j, Fk ∈ G | θk ),

The next theorem is the main result of this paper.
Theorem 5 Let pij = P (θ1 = j | θ0 = i) for all
i, j ∈ {1, . . . , N }. For all 0 ≤ s ≤ t < ∞, define the
Rn×n -valued random variable Φ(t; s) by the differential
equation ∂Φ/∂t = Aσt Φ(t; s) with the initial conditions
Φ(t; t) = In for every t ≥ 0. Assume that the following
two conditions hold:

and the conditional probability
P (θk+1 = j, Fk ∈ G | θk = i)

(6)

does not depend on k. The mean stability and the positivity of Σd is defined in the following standard manner. For a positive integer m, we say that Σd is exponentially mth mean stable if there exist C > 0 and β > 0
such that E[kxd (k)km ] ≤ Ce−βk kx0 km for all x0 and θ0 .
Also
P∞ we say thatmΣd is stochastically mth mean stable if
k=0 E[kxd (k)k ] is finite for all x0 and θ0 . Finally, Σd
is said to be positive if x0 ≥ 0 implies xd (k) ≥ 0 with
probability one for every k and θ0 .

(A1) Either m is even or Σ is positive;
(A2) There exists T > 0 such that τk+1 − τk ≤ T with
probability one for every k ≥ 0.
Then, Σ is exponentially mth mean stable if and only if
the (N nm ) × (N nm ) real block matrix A = [Aij ]1≤i,j≤N
with the (i, j)-block Aij being defined by

The next proposition relates the stability of Σ and SΣ.
Aij = pji E[Φ(τ1 ; 0)[m] | θ0 = j, θ1 = i]

(4)
Proposition 6 SΣ is a discrete-time semi-Markov
jump linear system. Moreover, if (A2) holds, then the
following statements are true:

is Schur stable.
In Section 3, we will show that the above theorem can recover stability characterizations of Markov jump linear
systems (Fang & Loparo 2002, Ogura & Martin 2014),
semi-Markov jump linear systems (Antunes et al. 2013,
Ogura & Martin 2014), and regenerative switched linear systems (Ogura & Martin 2015). We will also show
that the theorem can give a stability condition for other
switched linear systems whose stability cannot be studied using the methods in the literature.

• If Σ is exponentially mth mean stable, then SΣ is
stochastically mth mean stable.
• If SΣ is exponentially mth mean stable, then so is Σ.
PROOF. Let Fk = Φ(τk+1 ; τk ). Also, we let (Ω, M, P )
denote the underlying probability space. We denote the
σ-algebra of Ω generated by a set X of random variables

3

by M(X). We define the following σ-algebras; M1 =
M({θk }), M2 = M({θk , . . . , θ0 , Fk−1 , . . . , F0 }), M3 =
M({θk , . . . , θ0 , σs ; s ≤ τk }), and M4 = M({θk , σs ; s ≤
τk }). Then, we can show
M1 ⊂ M2 ⊂ M3 = M4 .

convex cone having a nonempty interior and satisfying
K ∩ (−K) = {0},

(8)

where −K := {−x : x ∈ K}. K is clearly a closed and
convex cone.

(7)

The first inclusion is obvious. The second inclusion is
true because F0 , . . . , Fk−1 are measurable on M3 . Finally, the last identity follows from (P2). Now, from (P3)
we know that P (θk+1 = j, Fk ∈ G | M1 ) = P (θk+1 =
j, Fk ∈ G | M4 ). Therefore, by (7) and Ogura & Martin
(2014, Lemma 1.1), we conclude that P (θk+1 = j, Fk ∈
G | M1 ) = P (θk+1 = j, Fk ∈ G | M2 ), which is equivalent to (5). Also, the conditional probability (6) is independent of k by (P3) and the time-homogeneity of θ.
Therefore, SΣ is a discrete-time semi-Markov jump linear system.

Let us show (8). For each i ∈ {1, . . . , n}, let `i denote
[m]
, i.e.,
the position of the monomial xm
i in the vector x
[m]
m
we assume that (x )`i = xi . Let us show the existence
of a constant C > 0 such that every y ∈ K satisfies
y`i ≥ 0, |y` | ≤ C

n
X

y `i ,

(9)

i=1

for all 1 ≤ i ≤ n and 1 ≤ ` ≤ nm . Take an arbitrary y in the convex hull of (Rn )[m] . Then there exist
x1 , . . . , xN ∈ Rn and positive numbers c1 , . . . , cN such
PN
[m]
that y =
j=1 cj xj . Without loss of generality we
PN
m
can assume cj = 1. Then, y`i =
j=1 (xj )i ≥ 0 because m is even. Next, let 1 ≤ ` ≤ nm be arbitrary
and let α = (α1 , . . . , αn ) be the exponent of the√monomial (x[m] )` , i.e., we suppose that (x[m] )` = α!xα .
Then, the inequality of arithmetic
√ Pnand geometric means
implies |(x[m] )` | ≤ m−1 α! i=1 αi xm
. Hence, the
Pi N
[m]
triangle inequality shows that |y` | ≤
j=1 |(xj )` | ≤
√
P
P
P
n
N
n
m−1 α! i=1 j=1 (xj )m
i ≤ C
i=1 y`i , where C =
√
m−1 maxα ( α!). Therefore, every y in the convex hull
of (Rn )[m] satisfies (9). A limiting argument thus shows
that (9) is satisfied by every y ∈ K. Now assume
y, −y ∈ K. This implies y`i = 0 for every i and therefore
y` = 0 for every ` by (9). Hence (8) holds.

The proof of the second statement can be done in the
same way as the proof for the implication [2 ⇒ 3] of
Ogura & Martin (2015, Theorem 12) due to (A2) and the
assumption (1). Also, we can prove the third statement
in the same way as the proof for [3 ⇒ 1] of Ogura &
Martin (2015, Theorem 12) by (A2). The details of the
proofs are thus omitted.
Proposition 6 shows that the stability analysis of Σ could
be reduced to the stability analysis of discrete-time semiMarkov jump linear systems. The next proposition gives
a characterization of the exponential mth mean stability of discrete-time semi-Markov jump linear systems,
extending Ogura & Martin (2014, Theorem 3.4) to the
case where m is even.
Proposition 7 For i, j ∈ {1, . . . , N }, let pij denote the
transition probability of θ from i to j. Assume that either
m is even or Σd is positive. Then, the following statements are equivalent.

Finally, by Ogura & Martin (2014, Lemma 1.5), the difference K − K = {x − y : x, y ∈ K} equals the whole
space Rnm . This in fact shows that the interior of K is
nonempty because, in general, a closed and convex cone
K satisfying (8) has a nonempty interior if and only if the
difference K − K coincides with the whole space (Tam
& Schneider 2006, Chapter 26).

(1) Σd is exponentially mth mean stable;
(2) Σd is stochastically mth mean stable;
(3) The (N nm ) × (N nm ) block matrix F with the (i, j)[m]
block Fij = pji E[F0 | θ0 = j, θ1 = i] ∈ Rnm ×nm
is Schur stable.

Now, since K is a proper cone, the N -direct product
K × · · · × K in RN nm is also a proper cone. Therefore
there exists (Seidman, Schneider & Arav 2005, Section 2)
a norm ||| · ||| on RN nm and a vector f ∈ RN nm having
the desired property. This completes the proof.

For the proof of this proposition, we will need the next
lemma.
Lemma 8 Assume that m is an even integer. Let K be
the closed convex hull of (Rn )[m] = {x[m] : x ∈ Rn } in
Rnm . Then, there exists a norm ||| · ||| on RN nm and f ∈
RN nm such that |||x||| = f > x for every x ∈ K × · · · × K.

We then prove Proposition 7:

PROOF of Proposition 7. The case where Σd is positive is shown in Ogura & Martin (2014, Theorem 3.4).
Let us assume that m is even. We shall show the cycle

PROOF. We first show that K is a proper cone (Tam
& Schneider 2006, Chapter 26), that is, K is a closed and

4

[1 ⇒ 2 ⇒ 3 ⇒ 1]. It is easy to prove [1 ⇒ 2]. The implication [2 ⇒ 3] can be proved in the same way as in the
proof of Theorem 3.4 in Ogura & Martin (2014) without
the positivity assumption.

Throughout this section, we will use the following notation (see Brockett 1973). For a matrix A ∈ Rn×n , we
define A[m] ∈ Rnm ×nm as the unique real matrix such
that
(exp(At))[m] = exp(A[m] t).
(13)

Let us prove [3 ⇒ 1] assuming m is even. Take a norm
||| · ||| on RN nm and f ∈ RN nm satisfying the linearity
property described in Lemma 8. By the equivalence of
the norms on a finite-dimensional linear space, we can
take a constant C1 > 0 such that C1−1 k·k ≤ ||| · ||| ≤
C1 k·k. Let us consider the stochastic process eθk , where
e1 , . . . , eN denote the standard unit vectors in RN . Using the general identity kx[m] k = kxkm (see Parrilo &
Jadbabaie 2008), we can show the inequality kxd (k)km =
kxd (k)[m] k = keθk ⊗ xd (k)[m] k ≤ C1 |||eθk ⊗ xd (k)[m] |||.
Since eθk ⊗ xd (k)[m] ∈ K × · · · × K, the linearity of the
norm ||| · ||| shows that
E[kxd (k)km ] ≤ C1 |||E[eθk ⊗ xd (k)[m] ]|||
≤ C12 kE[eθk ⊗ xd (k)[m] ]k.

3.1

In this subsection, we discuss some implications of
the stability characterization provided for regenerative switched linear systems (Ogura & Martin 2015).
We first recall the definition of regenerative processes.
We say that a stochastic process σ is a regenerative
process (Smith 1955) if there exists a random variable
R1 > 0 such that (i ) {σt+R1 }t≥0 is independent of
{{σt }t<R1 , R1 }, and (ii ) {σt+R1 }t≥0 has the same joint
distribution as {σt }t≥0 . Then, following Ogura & Martin (2015), we say that Σ given in (3) is a regenerative
switched linear system if σ is a regenerative process.

(10)

Let random variables {Rk }k≥1 be independent and identically distributed and define τk = R1 +· · ·+Rk for each
k ≥ 1. Then, σ can be broken into independent and identically distributed cycles {σt }0≤t<τ1 , {σt }τ1 ≤t<τ2 , . . .
(see Sigman & Wolff 1993). From this observation, it is
immediate to see (Çinlar 1975) that σ is Markov regenerative process with its embedded Markov renewal process
being (θ, τ ), where θ is the constant sequence {1, 1, . . . }.
Therefore, we can define the exponential mth mean stability of a regenerative switched linear system by Definition 3. We can furthermore prove, as a corollary of
Theorem 5, the following stability characterization for
regenerative switched linear systems originally given in
Ogura & Martin (2015, Theorem 12):

On the other hand, by the identity
E[eθk+1 ⊗ xd (k + 1)[m] ] = FE[eθk ⊗ xd (k)[m] ]

(11)

proved in Ogura & Martin (2014, Proposition 3.8), if
ρ(F) < 1 then there exist C2 > 0 and β > 0 such that
[m]

kE[eθk ⊗ xd (k)[m] ]k ≤ C2 e−βk keθ0 ⊗ x0 k
[m]

= C2 e−βk kx0 k
−βk

= C2 e

(12)

m

kx0 k .

Therefore, the inequalities (10) and (12) prove that Σd
is exponentially mth mean stable. This completes the
proof of Proposition 7.

Corollary 9 (Ogura & Martin (2015, Theorem 12))
Let Σ be a regenerative switched linear system. Assume
that (A1) is true and, moreover, there exists T > 0 such
that R1 ≤ T . Then, Σ is exponentially mth mean stable
if and only if the matrix E[Φ(R1 ; 0)[m] ] is Schur stable.

Now we can readily prove Theorem 5. (A1) implies that
either m is even or SΣ is positive. Therefore, by Proposition 7 and the first statement of Proposition 6, we see
that the following three properties are equivalent; the exponential mth mean stability of SΣ, the stochastic mth
mean stability of SΣ, and the Schur stability of A. This
equivalence and also the second and third statements of
Proposition 6 immediately prove the main result of Theorem 5.
3

Regenerative switched linear systems

PROOF. We first remark that, since R1 > 0, assumption (1) is satisfied. Moreover, R1 ≤ T guarantees that
(A2) is satisfied. Since (A1) is true by the assumption
in the corollary, we can apply Theorem 5. Since θ ≡ 1,
the matrix A given by (4) equals the nm × nm matrix
p11 E[Φ(τ1 ; 0)[m] | θ0 = 1, θ1 = 1] = E[Φ(R1 ; 0)[m] ].
Therefore, Theorem 5 readily completes the proof of the
corollary.

Applications

In this section, we illustrate how Theorem 5, the main
result of this paper, can be used to recover various stability characterizations of switched linear systems derived
in the literature (Subsections 3.1–3.3), as well as to analyze the stability of systems that cannot be analyzed
by current techniques in the literature (Subsection 3.4).

3.2

Semi-Markov jump linear systems

We now consider another class of switched linear systems called semi-Markov jump linear systems (Ogura &

5

pij (h) = P (θ1 = j | θ0 = i) = P (σh = j | σ0 = i). Then,
by Theorem 5, Σ is exponentially mth mean stable if and
only if the block matrix A(h) = [A(h)ij ]1≤i,j≤N given
by A(h)ij = pji (h)E[Φ(h; 0)[m] | σh = i, σ0 = j] is Schur
stable for every h > 0. We continuously extend the domain of A to the origin by letting A(0) = I. The continuity of the extended A at the origin can be verified by
the Dominant Convergence theorem. Then, to complete
the proof of the corollary, it is sufficient to show that

Martin 2014). Assume that (θ, τ ) is a Markov renewal
process taking values in {1, . . . , N } × [0, ∞), whose definition was given in the beginning of Section 2. Then,
the stochastic process {σt }t≥0 defined by
σt = θk , τk ≤ t < τk+1 ,

(14)

is called a semi-Markov process (Çinlar 1975). Then,
following Ogura & Martin (2014), we say that Σ given
by (3) is a semi-Markov jump linear system if σ is a
semi-Markov process. It is easy to see that σ is a Markov
regenerative process with its embedded renewal process
being (θ, τ ). The mapping π in (P2) is taken to be the
identity. Without loss of generality, we can assume (1).

A(h) = exp(BΣ h)

(15)

for every h ≥ 0.
To prove (15), we need to show that

The following corollary of Theorem 5, which extends
Ogura & Martin (2014, Theorem 2.5) to not necessarily positive systems, immediately follows from (13) and
(14):

A(h)A(h0 ) = A(h + h0 ) for all h, h0 ≥ 0

(16)

and
A0 (0) = BΣ .

Corollary 10 Suppose that Σ is a semi-Markov jump
linear system and assume that conditions (A1) and
(A2) hold. Then, Σ is exponentially mth mean stable
if and only if the (N nm ) × (N nm ) real block matrix
A = [Aij ]1≤i,j≤N with the (i, j)-block Aij being defined
by Aij = pji E[exp((Aj )[m] τ1 ) | θ0 = j, θ1 = i] is Schur
stable.
3.3

(17)

Let us show (16). Notice that, by (11), the trajectory x
of Σ satisfies
E[eσt+h ⊗ x(t + h)[m] ] = A(h)E[eσt ⊗ x(t)[m] ]

(18)

for all t ≥ 0 and h ≥ 0. This in particular im[m]
[m]
plies A(h + h0 )(eσ0 ⊗ x0 ) = A(h)A(h0 )(eσ0 ⊗ x0 )
n
for all σ0 ∈ {1, . . . , N } and x0 ∈ R . Since the set
[m]
{ei ⊗x0 : i ∈ {1 . . . , N } and x0 ∈ Rn } spans the whole
space RN nm by (Ogura & Martin 2014, Lemma 1.4), we
obtain (16).

Markov jump linear systems

In this subsection, we show that Theorem 5 can recover
stability characterizations (Ogura & Martin 2014, Fang
& Loparo 2002) for a class of switched linear systems
called Markov jump linear systems (Costa et al. 2013).
We say that Σ given by (3) is a Markov jump linear system if σ is a time-homogeneous Markov process. Generalizing the stability characterizations of Markov jump
linear systems in Fang & Loparo (2002, Theorem 3.3)
for m = 2 and in Ogura & Martin (2014, Theorem 5.1)
for positive systems, we can show the next corollary of
Theorem 5:

Then let us show (17). We compute A0ij (0), the (i, j)block of A0 (0), for each pair (i, j). First assume i 6=
j. Since limh→0 E[Φ(h; 0)[m] | σh = i, σ0 = j] = I,
by the Dominated convergence theorem, we can show
A0ij (0) = limh→0 (pji (h)I − O)/h = λji I. Next assume
i = j. Let γ denote the number of transitions of σ on
the interval [0, h] when σ0 = i. Then the event {σh =
i, σ0 = i, γ = 1} has probability zero. Moreover, using
the big-O asymptotic notation, we can show that

Corollary 11 Suppose that Σ is a Markov jump linear
system and σ can take values in {1, . . . , N }. Assume that
either m is even or A1 , . . . , AN are Metzler. Let Q be the
infinitesimal generator of Markov process σ. Then, Σ is
exponentially mth mean stable if and only if the matrix
LN
BΣ = Q> ⊗ Inm + i=1 (Ai )[m] is Hurwitz stable.

E[Φ(h; 0)[m] | σh = i, σ0 = i, γ = `]
 (Ai ) h
[m] ,
e
for ` = 0,
=
I + O(h), for ` ≥ 2,
P (γ = 0) = eλii h , and P (γ ≥ 2) = O(h2 ). Therefore

PROOF. Notice that, for h > 0 arbitrarily chosen,
σ is a Markov regenerative process with the embedded
Markov renewal process (θ, τ ) = {(σkh , kh)}k≥0 by the
Markovian property of σ. Therefore, Σ is a Markov regenerative switched linear system. The assumption (1)
is trivially true and, also, condition (A2) is satisfied
with T = h. Moreover, the assumption of the corollary ensures that condition (A1) also holds true. Let

Aii (h) − Aii (0)
pii (h)e(Ai )[m] h − I
=
+ O(h)
h
h
→ (Ai )[m] + λii I
as h → 0. The above argument proves (17) and, therefore, completes the proof of the corollary.

6

3.4

as well as (1) hold true. Therefore, by Theorem 5, it
is sufficient to show that the matrix A given by (4)
equals Ah . To compute Aij , observe that σ confined in
[0, h) with the condition r0 = j is a time-homogeneous
Markov process with state space {(1, j), . . . , (N, j)} and
infinitesimal generator Q. Therefore, from (15), Aij
equals the (i, j)-block of the matrix exp(Bj h). Hence,
the j-th block-column of A equals that of exp(Bj h).
This proves A = Ah , as desired.

Markov jump linear systems with periodic mode observation

In this subsection, we present an example of a system to which none of the results in Corollaries 9–11
are applicable. Consider the constant-gain statefeedback control of the Markov jump linear system dx/dt = AP,rt x(t) + BP,rt u(t), where AP,1 ,
. . . , AP,N ∈ Rn×n , BP,1 , . . . , BP,N ∈ Rn×p , and r
is a time-homogeneous Markov process with state
space {1, . . . , N } and infinitesimal generator Q. If the
controller can measure r at any time instant, one can
consider the feedback control
u = Kr x

Remark 13 Note that we can use neither Corollaries 9
nor 10 to analyze the stability of Σh . First, it is shown
in Ogura & Martin (2015, Example 7) that σ cannot
be a semi-Markov process. Moreover, though σ can be
realized as a regenerative process, the realization in Ogura
& Martin (2015, Example 7) does not satisfy (A2) by the
following reason. In the embedded renewal process (θ, τ )
of the realization, the renewal times {τk }k≥0 are given
as the sampling times t = kh (k = 0, 1, . . . ) such that
r(t) = r0 . However, the difference τk+1 − τk can take an
arbitrary large number and, hence, cannot be bounded by
a uniform and finite number.

(19)

with mode-dependent gains Ki ∈ Rp×n (i = 1, . . . , N ).
It is well known (Costa et al. 2013) that, under this ideal
situation, one can find feedback gains Ki that stabilize
the closed-loop system in the mean-square sense by solving linear matrix inequalities.
Following the formulation of Cetinkaya & Hayakawa
(2015) in discrete-time, we here consider a more realistic situation where only the periodic samples {rkh }k≥0
with a known sampling period h > 0 are available to
the controller. Precisely speaking, we assume that the
feedback control takes the form
u = Kq x

Let us see an example. Consider the Markov jump linear
system with the following coefficient matrices:




0
0
0
0
0
0







A1 = 
0 −0.545 0.626 , A2 = 0 −0.106 0.087 ,
0 −1.570 1.465
0 −3.810 3.861


1, 80 −0.3925 4.52


A3 = 
0.100 −0.28
 3.14
,
−19.06 −0.148 1.56






−0.064
0
0






 , B2 =  0  , B3 =  0.195  ,
B1 = 
−0.283






−0.080
0.087
0.333

(20)

where the stochastic process q is defined by qt = rkh if
kh ≤ t < (k + 1)h for all t ≥ 0 and k ≥ 0. We emphasize
that, though we assume that only periodic samples of r
are available, the infinitesimal generator Q is assumed to
be known before the feedback control is designed. Defining σ = (r, q) and Aσ = AP,r + BP,r Kq , we can write
the closed-loop equation in the form (3), which we denote by Σh . Since r is time-homogeneous and Markovian, we can see that σ is a Markov regenerative process
with an associated embedded Markov renewal process
{(rkh , kh)}k≥0 . The function π in (P2) maps the pair
(i, j) ∈ {1, . . . , N }2 to i. Then, using Theorem 5, we can
prove the following characterization of the exponential
mean stability of Σh :

and the infinitesimal generator

Corollary 12 Assume that m is even. For each j ∈
LN
{1, . . . , N }, let Bj = Q> ⊗Inm + i=1 (A(i,j) )[m] . Define
Ah =

N
X

exp(Bj h)((ej e>
j ) ⊗ Inm ).



−0.53 0.32 0.21



Q=
 0.50 −0.88 0.38  .
0.40 0.13 −0.53

(21)

j=1

Then, Σh is exponentially mth mean stable if and only if
Ah is Schur stable.

This system, taken from Blair & Sworder (1975), models
a certain economic system. We denote by Σ0 the closedloop system when the classical feedback control (19) is
applied to this system. In Costa et al. (2013), the feedback gains that stabilize Σ0 (in the mean square sense)

PROOF. Since m is even, condition (A1) is satisfied. Moreover, because τk+1 − τk = h, condition (A2)

7

5
4
3

kx(t)k2

h−1 log ρ(Ah )

100

2
1
0
-1
0

10
0.1

0.169 0.2

0.3

-4

0

1

2

3

t

h

(a) h = 0.1 (stable)

Fig. 1. h−1 log(ρ(Ah )) versus h
104

with the minimum H 2 norm are obtained as
kx(t)k2

h
i
K1 = 2.0343 14.5181 −23.5917 ,
h
i
K2 = 1.0187 73.0961 −78.7596 ,
h
i
K3 = 93.6651 −11.4921 11.6875 .

100

10-4
0

1

2

3

t

We use Corollary 12 to investigate how the stability
property of Σ0 is altered when the feedback control (19)
is replaced with the one in (20) based on periodic
observations. For m = 2 and period h = (0.001)`
(` = 1, . . . , 300), we compute the spectral radius of Ah
given in Corollary 12. In Fig. 1, we show the graph
of h−1 log(ρ(Ah )) as h varies. From the corollary and
the graph, we determine that the system Σh is mean
square stable if and only if 0 < h < 0.169 [years]. This
implies that, to guarantee the stability of the controlled
economic system, r must be sampled with a period less
than about 2 months. It is interesting to observe that,
as h → 0, the quantity h−1 log(ρ(Ah )) becomes close to
−0.250, the maximum real part of the eigenvalue of the
matrix BΣ0 that characterizes the stability of the original system Σ0 by Corollary 11. This shows that, in the
limit of h → 0, the stability of the original closed-loop
system Σ0 is “recovered” by Σh .

(b) h = 0.3 (unstable)
Fig. 2. Sample paths of kx(t)k2 (thin lines) and their averages
(thick lines) for h = 0.1 and 0.3

3

r
q

2
1
0

1

2

3

t

(a) h = 0.1 (stable)
3

r
q

2

In Figs. 2a and 2b, we show 100 sample paths of kx(t)k2
and their sample averages when h = 0.1 and h = 0.3,
respectively. For the generation of each sample path, we
take x0 randomly from the unit sphere in R3 and r0 from
the uniform distribution on {1, 2, 3}. We can see that
mean square stability is achieved with sampling period
h = 0.1, while the closed-loop system exhibits instability
for h = 0.3. Finally, for each value of h, we show sample
paths of σ = (r, q) in Fig. 3. We notice that the change of
the values of q occurs only at sampling instants, shown
by the dotted lines.

1
0

1

2

3

t

(b) h = 0.3 (unstable)
Fig. 3. Sample paths of σ = (r, q) for h = 0.1 and 0.3. Solid
lines, dashed lines, and dotted lines show r, q, and sampling
instants, respectively.

8

4

Conclusion

Sigman, K. & Wolff, R. W. (1993), ‘A Review of regenerative
processes’, SIAM Review 35, 269–288.
Smith, W. L. (1955), ‘Regenerative stochastic processes’,
Proceedings of the Royal Society A: Mathematical, Physical
and Engineering Sciences 232(1188), 6–31.

In this paper, we have investigated the mean stability of
Markov regenerative switched linear systems. The class
of switched linear systems contain a wide variety of important stochastic switched linear systems that have appeared in the literature. We have shown that the mean
stability of a Markov regenerative switched linear system is characterized by the spectral radius of a matrix
arising from its transition matrix. A numerical example
was presented to illustrate the obtained result.

Tam, B.-S. & Schneider, H. (2006), Matrices Leaving a Cone
Invariant, in L. Hogben, ed., ‘Handbook of Linear Algebra’,
Chapman & Hall/CRC, chapter 26.

References
Antunes, D. J., Hespanha, J. P. & Silvestre, C. J. (2013),
‘Stochastic hybrid systems with renewal transitions: moment
analysis with application to networked control systems
with delays’, SIAM Journal on Control and Optimization
51, 1481–1499.
Blair, W. P. & Sworder, D. D. (1975), ‘Continuous-time regulation
of a class of econometric models’, IEEE Transactions on
Systems, Man, and Cybernetics SMC-5, 341–346.
Brockett, R. W. (1973), ‘Lie theory and control systems defined
on spheres’, SIAM Journal on Applied Mathematics 25, 213–
225.
Çinlar, E. (1974), ‘Periodicity in Markov renewal theory’,
Advances in Applied Probability 6(1), 61–78.
Çinlar, E. (1975), ‘Exceptional paper–Markov renewal theory: A
survey’, Management Science 21(7), 727–752.
Cetinkaya, A. & Hayakawa, T. (2015), ‘Feedback control of
switched stochastic systems using randomly available active
mode information’, Automatica 52, 55–62.
Choi, H., Kulkarni, V. G. & Trivedi, K. S. (1994), ‘Markov
regenerative stochastic Petri nets’, Performance Evaluation
20, 337–357.
Costa, O. L. V., Fragoso, M. D. & Todorov, M. G. (2013),
Continuous-time Markov Jump Linear Systems, Springer.
Distefano, S. & Trivedi, K. S. (2013), ‘Non-Markovian state-space
models in dependability evaluation’, Quality and Reliability
Engineering International 29, 225–239.
Fang, Y. & Loparo, K. (2002), ‘Stabilization of continuoustime jump linear systems’, IEEE Transactions on Automatic
Control 47(10), 1590–1603.
Farina, L. & Rinaldi, S. (2000), Positive Linear Systems: Theory
and Applications, Wiley-Interscience.
Hou, Z., Luo, J. & Shi, P. (2005), ‘Stochastic stability of
linear systems with semi-Markovian jump parameters’, The
ANZIAM Journal 46, 331–340.
Ogura, M. & Martin, C. F. (2014), ‘Stability analysis of positive
semi-Markovian jump linear systems with state resets’, SIAM
Journal on Control and Optimization 52, 1809–1831.
Ogura, M. & Martin, C. F. (2015), ‘Stability analysis of linear
systems subject to regenerative switchings’, Systems &
Control Letters 75, 94–100.
Parrilo, P. A. & Jadbabaie, A. (2008), ‘Approximation of the
joint spectral radius using sum of squares’, Linear Algebra
and its Applications 428, 2385–2402.
Seidman, T. I., Schneider, H. & Arav, M. (2005), ‘Comparison
theorems using general cones for norms of iteration matrices’,
Linear Algebra and its Applications 399, 169–186.

9

