The continuity and uniqueness of the value function of the hybrid opti
mal control problem with reach time to a target set
Myong-Song Ho
Kim Il Sung University, Pyongyang DPRK
ryongnam5@yahoo.com
Kwang-Nam O
Middle School No.1, Pyongyang DPRK
ryongnam6@yahoo.com
Chol-Jun Hwang
Kim Il Sung University, Pyongyang DPRK
ryongnam27@yahoo.com
Abstract: The hybrid optimal control problem with reach time to a target set is addressed and the continuity and u
niqueness of the associated value function is proved.
Hybrid systems involves interaction of different types of dynamics: continuous and discrete dynamics. The state of
a continuous system is evolved by an ordinary differential equation until the trajectory hits the predefined jump sets: an
autonomous jump set A and a controlled jump set C . At each jump the trajectory is moved discontinuously to another
Euclidean space by a discrete system.
We study the hybrid optimal control problem with reach time to a target set, prove the continuity of the associated
value function V with respect to the initial point under the assumption that V is lower semicontinuous on the boundary
of a target set, and also characterize it as an unique solution of a quasi-variational inequality in a viscosity sense using
the dynamic programming principle.
Keyword: hybrid control, quasi-variational inequality, viscosity solution, dynamic programming principle, uniqueness

1. Introduction
Hybrid control systems are control systems that involve both continuous and discrete dynamics
and continuous and discrete controls. The hybrid system has been studied widely in the number of li
teratures and also has been applied successfully to address the problems in air traffic control [4], aut
omotive control [6], chemical process control [7] and so on.
In [5], Branicket, Borkar, and Mitter introduced an unified model for hybrid control. They have
specified priori regions Ai , Ci , Di , ( i ∈ Z + ) , namely , an autonomous jump set, a controlled jump set,

and a jump destination set. In their work the state evolves according to an ordinary differential equation in some state space and when the trajectory hits a jump set the state is jumped to a destination
set D according one of the two possibilities: if the trajectory hits an autonomous jump set, then the
state must jump while the trajectory hits a controlled jump set , the controller chooses to (it does not
have to) move the trajectory discontinuously.
They proved a right continuity of the value function of infinite horizon optimal control problem
and derived quasi-variational inequality satisfied by the value function and summarized algorithms
for solving this inequalities.
In [1], Dharmatti and Ramaswamy studied the same problem as the one considered in [5] in the
most general case when the autonomous jump set A is nonempty and the controlled jump set can be
arbitrary, proved the local Holder continuity of the value function, and demonstrated that this valu
e function is an unique solution of the quasi-variational inequality (QVI) in a viscosity sense when i
t is bounded.
To prove the continuity they estimated the distances between the trajectories starting from two
closer points, through their continuous evolutions, and through their discrete jumps to show that the
1

distance remains small for initial points sufficiently close enough. This allowed to getting a Holder
exponent for the continuity of the value function.
They also considered in [2] the finite horizon hybrid control problem and proved continuity and
uniqueness of the value function in the case that the cost functionals are assumed to be unbounded.
In this paper we observe the hybrid optimal control problems with reach time to a target set.
Unlike the infinite horizon control problem which is noted in [1], in our hybrid optimal control
problem the evolution of the state is stopped and the payoff is computed when the trajectory hits the
target set defined in some state space.
Our aim is to prove the continuity of the value function and to characterize the value function
V as the unique viscosity solution of the quasi-variational inequality.
The proof of continuity of the value function in this case is harder than in infinite horizon hybrid
optimal control problem because of non-zero terminal cost h .
In order to have a continuous solution for the value function we have to assume that V is lower
semicontinuous on the boundary of the target set Γ .
Under this assumption we prove that the continuity at all points on ∂Γ implies continuity every
where.
Although the basic estimations are similar to those considered in [1], we shoud take care to the
evaluation of the differences of the hitting times in the proof of the continuity.
In [1], they defined the first hitting time of the trajectory as T ( x ) = inf{t > 0 | X x ( t , u ) ∈ A} ,
u

( X x ( t , u ) is the state of the trajectory starting from an initial point x and evolving with continuous
control u ), and proved that this is locally Lipschitz continuous (in Theorem 3. 1) to investigate the
distance between the first hitting points (Lemma 3. 2).
On the other hand in the proof of Theorem 3. 5 only two types of integrals are considered to sum
up the total costs from the conclusion that the trajectories starting from two neighbouring points hit
the autonomous jump set sequentially (expression 3. 22).
When taking into consideration the problem of our own style, instead of using T ( x ) we define
the first hitting time of the trajectory starting from the initial point x with control u to evaluate the
differences between two trajectories.
What matters in regard to us in the proof of the continuity is that the claculating of the costs in
various time intervals because it doesn`t seem to be suitable to use the above relations between the
hitting times (expression 3. 22) directly by means of our problem.
This paper is organized as follows:
In section 2 we introduce the notations to define the hybrid control system and associated reach
time problem to a target set and elucidate all assumptions used in deriving the results of the sequel.
In section 3 we prove the continuity of the value function with respect to the initial point under
the assumption that the value function is lower semicontinuous on the boundary of a target set.
Section 4 deals with the dynamic programming principle for the present problem and the quasivariational inequality that the value function must be satisfied in the viscosity sense.
In section 5 we prove the comparison between any two viscosity solutions to take up the issue o
f uniqueness.
2. Mathematical preliminaries.

In this section we define the hybrid optimal control system and give all assumptions which are
needed for the consideration of the continuity and uniqueness of the value function.
Hybrid control systems are dynamical systems that involve the interaction of different types of
dynamics: continuous flows and discrete jumps. Therefore the state of the hybrid control system is
described by a pair of the variables: a part of the state takes values in Euclidean space while another
part takes values in a discrete set.
2

The evolution of the state of the continuous system is given as a following ordinary differential
equation
y& ( t ) = f ( y ( t ) , q ( t ) , u ( t ) )
(2. 1)
y (0) = x ,

(2. 2)

Here y ( t ) ∈ Ω, Ω = U Ωi with Ωi is a closed connected subset of R di , and q ( t ) ∈ I = {1, 2, L}
i

is a discrete variable that is q ( t ) = i whenever y ( t ) ∈ Ωi . At this time (2. 1) can be written in another
form

y& ( t ) = fi ( y ( t ) , u ( t ) ) ,

where fi : Ωi × u → Ωi and

u

is control set

u = {a :[0, ∞) → U | u (⋅) : measurable, U : compact metric space } .
Denote the state of the hybrid control system at time t by ( y x ( t , u ) , q ( t ) ) .
Here y x ( t , u ) is a solution of (2. 1), (2. 2).

In the hybrid control system the trajectory undergoes discrete jump to a destination set D when
it hits predefined autonomous jump set A or controlled jump set C .
A = U Ai , Ai ⊂ Ωi ⊂ R di
i

C = U Ci , Ci ⊂ Ωi ⊂ R di
i

D = U Di , Di ⊂ Ωi ⊂ R di
i

When the trajectory starting from the initial state ( x, q ) hits the autonomous jump set Aq then
it is moved to a destination set Dq′ according to the transition map g . This map uses discrete contro
ls to move the trajectory.
When the trajectory hits the controlled jump set Cq , every time the controller can choose either
to jump or not to jump and if the controller chooses to jump then the trajectory is moved to a destina
tion set Dq′ possibly in another space Ω q′ to continue the evolving under the continuous dynamics
until it hits again the jump sets or a target set.
Then we get a sequence of hitting times of A (we denote it by {τ i } ) and a sequence of jumping
times of C , where the controller chooses to make a jump (we denote it by {ξi } ).

(

)

Denote the state that the trajectory hits A at time τ i by ( xi , qi ) = y (τ i − , u ) , q (τ i − ) , (in gener

al, we take the trajectory to be left continuous) and the shifted state under the transition map
g : A × I × V1 → D by xi′ , qi′ = y (τ i + , u ) , q (τ i + ) .

) (

(

)

Also for the controlled jump set C we denote the state where the controller chooses to jump at
time ξi by ( y (ξi ) , q (ξi ) ) = y (ξi − , u ) , q (ξi − ) , and jumped state by

(

( y (ξ )′ , q (ξ )′ ) = ( y (ξ , u ) , q (ξ

)

+

i

i

i

+
i

)) .

Now we give the following topology on Ω × I .
( xn , qn ) converges to ( x, q ) if for any ε > 0 , there exist some large number N and some i su
ch that qn = q, x, xn ∈ Ωi ⊆ Ri and xn − x < ε for ∀n ≥ N . Here ⋅ means a Euclidean norms in
R di

3

Throughout the whole paper we deal with the sets A, C , D and the map f , g satisfying follow
ing assumptions.
( A1 )
For any i ∈ I , Ωi is a closure of opened connected subset of R di .

( A2 )
( A3 )

Ai , Ci , Di are closed, ∂Ai , ∂Ci are C 2 , and ∂Ai ⊇ ∂Ωi for any xi ∈ Di .
The transition map g : A × I × V1 → D is bounded, Lipschitz continuous with Lipsch

itz constant G , with the understanding g = {gi } : if q ( t ) = i then the transition map in Ai is

regarded as gi : Ai × V1 → D j , where V1 is discrete control set.

Vector field f : Ω × I × u → Ω is a Lipschitz continuous with Lipschitz constant L i
( A4 )
n the state variable x , uniformly continuous in control variable u , and for any i ∈ I ,
f i ( x, u ) ≤ F , ∀x ∈ Ωi , u ∈ U
( A5 )

(transversality condition) For any i ∈ I , Ai is compact and there exist some constant

ξ0 > 0 such that fi ( x0 , u ) ⋅η ( x0 ) ≤ −2ξ 0 , ∀x0 ∈ ∂Ai , u ∈ U , where η ( x ) is an outward norm

al vector to ∂Ai at x .
We can give the same assumption to the controlled jump set C .
( A6 )
inf d ( Ai , Ci ) ≥ β > 0, inf d ( Ai , Di ) ≥ β > 0 , where d ( A, B ) is a distance between t
he sets A and B .
U , V1 are the compact metric spaces.
( A7 )
As a matter of fact, all the assumptions above are the same as those in [1].
Next for some j ∈ I we define a set Γ ⊂ Ω j , namely, a target set. This set satisfies the followin
g assumptions.
( A8 )
Γ ⊂ Ω j is a closed, ∂Γ is compact, and
d ( Γ, D j ) ≥ β > 0, d ( Γ, Aj ) ≥ β > 0, d ( Γ, C j ) ≥ β > 0

( A9 )
the transversality condition is satisfied on the target set Γ .
In this paper we observe the following hybrid optimal control problem with reach time to a targ
et set Γ for the control u ( ⋅) , v, ξ , y (ξ )′ .

(

i

The total discount is given by

(

i

)

J x, q, u ( ⋅) , v, ξi , y (ξi )′ =
+

∑
ξ
i <t x

(u )

(

)

tx (u )

λτ
λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + τ ∑( ) C ( x , q , v )e
−

− t

x

0

)

i <t x u

(

a

i

)

Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e − λξi + e − λtx ( u ) h yx ( t x ( u ) , u ) ,

i

i

(2. 3)

where λ is a positive constant and
K : Ω × I × u → R,
Ca : A × I × V1 → R,
Cc : C × I × D × I → R,
h : Γ → R,
are cost functions.
Unlike the problem in [1], our problem has an initial state in Ω \ Γ whose dynamics stopped an
d the payoff computed when the trajectory reaches Γ , therefore we define a reach time for u ∈ u
{t | y x ( t , u ) ∈ Γ} = φ
⎧⎪+∞
(2. 4)
tx ( u ) = ⎨
⎪⎩ min{t | y x ( t , u ) ∈ Γ} {t | yx ( t , u ) ∈ Γ} ≠ φ
The value function V of this problem is given as
4

(

)

inf
J x, q, u ( ⋅) , v, ξi , y (ξi )′ .
u
Here the following assumptions are satisfied for h, K , Ca , Cc .
V ( x, q ) =

( A10 )

θ ∈( ×V1 ×[ 0, +∞ )× D )

(2. 5)

h ∈ C ( Γ ) , h ( x ) ≥ 0, x ∈ Γ

( A11 )
K is a Lipschitz continuous function with Lipschitz constant in the state variable x
and uniformly continuous in the control variable u . Also
K ( x , q , u ) ≤ K 0 , ∀ ( x, q , u ) ∈ Ω × I × U
Ca , Cc are uniformly continuous in the variables x, u and bounded below. Moreover
( A12 )
Ca is a Lipschitz continuous function with Lipschitz constant C1 in the state variable x and

Ca ( x, q, v ) ≤ C0 , ∀ ( x, q, v ) ∈ Ω × I × V1 . Also
Cc ( x, q, y, q′ ) < Cc ( x, q, z , q′ ) + Cc ( z , q′, y, q′ ) , ∀x ∈ Cq , z ∈ D ∩ Cq′ , y ∈ Dq′

Now we define the first hitting time with control u and singed distance function
t ( x, q, u ) = inf{t > 0 | y ( t ) ∈ Aq , y ( 0 ) = x, y& ( t ) = f q ( y ( t ) , u ( t ) )}

⎧−d ( x, ∂Aq ) x ∈ int Aq
⎪⎪
d ( x, q ) = ⎨ 0
x ∈ ∂Aq .
⎪
⎪⎩ d ( x, ∂Aq ) x ∈ outAq
Here int Aq = Aq \ ∂Aq and outAq = Ω q \ Aq .
Remark1. The first hitting time we defined above is a different concept with the first hitting time
defined in the literature [1]. (See section 3 of [1], 1263 p.)
In the sequel discussion we replace ( y x ( t , u ) , q ( t ) ) , t ( x, q, u ) , d ( x, q ) simply as

y x ( t , u ) , t ( x, u ) , d ( x ) , while the discrete jump to another space is not considered.

In next sections we are focused on the continuity of value function of a hybrid optimal control
problem described above and the partial differential equation satisfied by the value function.
3. Continuity of the value function.

From the ordinary differential equation theory we can see the system whose dynamics is given i
n (2. 1), and (2. 2) has the following property
yx ( t , u ) − yz ( t , u ) < e Lt x − z
(3. 1)
yz ( t , u ) − yz ( t , u ) < F t − t

(3. 2)

Here F , L are the constants given in assumption ( A4 ) respectively.
Without loss of generality, let the initial points x, z is in Ω1 and fix the control u .
For the proof of the value function we consider how the first hitting times with control u depe
nds on the initial points.
Theorem1: Assume ( A1 )-( A7 ) to be satisfied. Let the first hitting times of trajectories starting

from x, z and moving with the fixed control u be τ 1 = t ( x, u ) , λ1 = t ( z , u ) and the points where t

he trajectories hit A be x1 = y x (τ 1 , u ) , z1 = yz ( λ1 , u ) , x1 , z1 ∈ A1 .
If x − z < δ1 , for some δ1 > 0 , then
x1 − z1 < (1 + FC ) e L(τ1 ∨ λ1 ) x − z ,
where a ∨ b = max{a, b}
5

(3. 3)

(Proof) First, we show the following lemma for the evaluation of τ 1 − λ1 .
Lemma1: there exist some δ * > 0, C > 0 such that for the control u

t ( x, q, u ) < Cd ( x, q ) , ∀x ∈ B ( Aq , δ * ) \ int Aq .

Here B ( Aq , δ * ) is a neighborhood of Aq and d is a signed distance function.
(Proof of lemma) Let the trajectory starting from x hit on Aq at x0 .

From the assumption ( A2 ), there exist some R > 0 such that d is C1 in B ( ∂Aq , R ) .

Now we show that there exist some r0 < R such that the following inequality holds for any

0 ≤ t < t ( x, q, u ) and yx ( t , u ) ∈ B ( Aq , r0 ) :

f q ( y x ( t , u ) , u ( t ) ) ⋅ Dx d ( y x ( t , u ) , q ) < −ξ 0 .

In fact from the transversality condition ( A5 ) it easy to know that for any x0 ∈ ∂Aq and u0 ∈ U ,
there exist some r0 < R such that

f ( x, u0 ) ⋅ Dx d ( x ) < −ξ 0 , ∀x ∈ B ( x0 , r0 ) ,

(*)

where Dx d ( x, q ) means the derivatives of d by means of the spatial variable x .
We can give the r0 in above inequality regardless of x0 .
As a matter of fact, whenever we fix the control u ∈ U , we can choose xi ∈ ∂Aq , i = 1, 2, L , n
and corresponding ri > 0 ( i = 1, 2, L , n ) such that

n

⎛

ri ⎞

U B ⎜⎝ x , 2 ⎟⎠ ⊃ ∂A
i

q

and

i =1

f ( x, u0 ) ⋅ Dd ( x ) < −ξ 0 , ∀x ∈ B ( xi , ri ) , ∀i = 1, n
by the compactness of ∂Aq .
r
r r
Lets r0 = min{ 1 , 2 , L , n } . If x ∈ B ( ∂Aq , r0 ) , then there exist some x′ ∈ ∂Aq and some
2 2
2
r ⎞
⎛
j ∈ {1, 2, L , n} such that x′ ∈ B ⎜ x j , j ⎟ and x − x′ < r0 .
2⎠
⎝
From this we can get x − x j ≤ x − x′ + x′ − x j ≤ rj which means (*) holds for every

x ∈ B ( ∂Aq , δ * ) that is

f ( x, u0 ) ⋅ Dd ( x ) < −ξ 0 , ∀x ∈ B ( ∂Aq , δ * ) ,

where δ * = min{r0 ,

(**)

β

}.
2
If there exist some x ∈ B ( ∂Aq , r0 ) and some t ′ ( 0 ≤ t ′ < t ( x, q, u ) ) such that
f q ( y x ( t , u ) , u ( t ) ) ⋅ Dx d ( y x ( t , u ) , q ) ≥ −ξ 0

for y x ( t ′, u ) ∈ B ( Aq , δ * ) , then it contradicts to (**) because it does not hold for
x = y x ( t ′, u ) , u0 = u ( t ′ ) .

Therefore if y x ( t , u ) ∈ B ( Aq , δ * ) for any 0 ≤ t < t ( x, q, u ) then

f q ( y x ( t , u ) , u ( t ) ) ⋅ Dx d ( y x ( t , u ) , q ) < −ξ 0 .
s

It means that d ( y x ( s, u ) , q ) − d ( x, q ) = ∫ f q ( y x ( t , u ) , u ( t ) ) ⋅ Dx d ( yx ( t , u ) , q ) dt < −ξ 0 s
0

6

Now if we choose sx =

d ( x, q )

ξ0

, then

d ( y x ( sx , u ) , q ) − d ( x, q ) < − d ( x, q ) , and this implies y x ( sx , u ) ∈ int Aq

Therefore for C =

1

ξ0

, we have t ( x, u ( t ) ) < sx =

d ( x, q )

ξ0

.

□

Now we apply the lemma to get (3. 3).
Without loss of generality let λ1 > τ 1 .
Define δ1 = e − Lτ1δ * where δ * is listed in the Lemma.
If x − z < δ1 , then from (3. 1)
y x (τ 1 , u ) − y z (τ 1 , u ) < e Lτ1δ *e − Lτ1 < δ *
Hence by using the lemma and y x (τ 1 , u ) ∈ A we have

λ1 − τ 1 = t ( yz (τ 1 , u ) , u ) < Cd ( yz (τ 1 , u ) ) < C yx (τ 1 , u ) − yz (τ 1 , u ) < Ce Lτ x − z
1

If λ1 < τ 1 then we have τ 1 − λ1 < Ce Lλ1 x − z in a similar way, therefore

τ 1 − λ1 < Ce L(τ ∨ λ ) x − z
1

(3. 4)

1

On the other hand
x1 − z1 < y x (τ 1 , u ) − yz ( λ1 , u ) ≤ yx (τ 1 , u ) − y z (τ 1 , u ) + y z (τ 1 , u ) − y z ( λ1 , u )
< e Lτ1 x − z + F τ 1 − λ1 ≤ e L(τ1 −λ1 ) (1 + FC ) x − z .
In the third inequality we used (3. 1) and (3. 2) and last inequality is from (3. 4)
Now without loss of generality let trajectories be moved from x1 , z1 to

□

x1′ = g1 ( x1 , v ) , z1′ = g1 ( z1 , v ) which belong to the initial set of D2 ⊂ Ω 2 and the evolving is going

on Ω 2 with fixed control u .
Denote the next hitting times of trajectories when they hit A once again by τ 2 , λ2 .
The following theorem estimates τ 2 − λ2 .
Theorem 2: Assume (2. 1)-(2. 7). Let the next hitting times of trajectories starting from x, z a
nd evolving with fixed control u be τ 2 , λ2 .

Define x2 = yx ′ (τ 2 − τ 1 , u ) , z2 = y z ′ ( λ 2 − λ1 , u ) .
1

1

Then there exist some δ 2 > 0 such that if x − z < δ 2 , then

( FC (1 + G ) + G ) x − z ,
< ( FC + 1) e (τ λ ) ( FC (1 + G ) + G ) x − z

τ 2 − λ2 < Ce(τ
x2 − z2

2 ∨ λ2

)

L

2∨ 2

(3. 5)
(3. 6)

Remark2: The Theorem 2 is also found in the literature [1], here we note that the first hitting ti
me T ( x ) in the proof of Lemma 3 of [1] is different with t ( x, u ) in this Theorem.
(Proof) without loss of generality let τ 1 < λ1 .

(

)

Then t yx ′ ( λ1 − τ 1 , u ) , u = τ 2 − λ1 and if yx ′ ( λ1 − τ 1 , u ) − z1′ < δ1 for δ1 > 0 which is defined
1

1

as in the Theorem 1, then

τ 2 − λ2 < Ce L(τ

2 − λ1 ∨ λ2 − λ1

)

yx ′ ( λ1 − τ 1 , u ) − z1′
1

by using (3. 4).
In the right side of above inequality
7

yx ′ ( λ1 − τ 1 , u ) − z1′ ≤ yx ′ ( λ1 − τ 1 , u ) − x1′ + x1′ − z1′
1

1

and by using (3. 2), (3. 4) again we have
yx ′ ( λ1 − τ 1 , u ) − x1′ ≤ F τ 1 − λ1 < FCe Lλ1 x − z .
1

Also
x1′ − z1′ < G x1 − z1 < G (1 + FC ) e Lλ1 x − z ,
by using the Lipschitz continuity of g and (3. 3).
Hence combining two inequalities, we get
yx ′ ( λ1 − τ 1 , u ) − z1′ < ( FCe Lλ1 + G (1 + FC ) e Lλ1 ) x − z
1

= e Lλ1 ( FC (1 + G ) + G ) x − z

Now we define δ 2 = min{δ1 ,

(3. 7)

e − Lλ1δ1
}.
FC (1 + G ) + G

If x − z < δ 2 , then yx ′ ( λ1 − τ 1 , u ) − z1′ < δ1 which implies (3. 5) is proved.
1

Also
x2 − z2 = y x ′ (τ 2 − τ 1 , u ) − y z ′ ( λ2 − λ1 , u )
1

1

≤ y x ′ (τ 2 − τ 1 , u ) − y z ′ (τ 2 − λ1 , u ) + y z ′ (τ 2 − λ1 , u ) − y z ′ ( λ2 − λ1 , u )
1

= yy
<e

1

x1′

( λ1 −τ1 )

L (τ 2 − λ1 )

1

(τ 2 − λ1 , u ) − yz ′ (τ 2 − λ1 , u ) +
1

1

yz ′ (τ 2 − λ1 , u ) − y z ′ ( λ2 − λ1 , u )
1

1

y x ′ ( λ1 − τ 1 , u ) − z1′ + F τ 2 − λ2
1

( FC (1 + G ) + G ) x − z + CFe (τ λ ) ( FC (1 + G ) + G ) x − z
≤ e (τ λ ) ( FC + 1) ( FC (1 + G ) + G ) x − z ,

<e

Lτ 2

L

L

2∨ 2

2∨ 2

where we used the semigroup property y x ′ (τ 2 − τ 1 ) = y y
1

x1′

( λ1 −τ1 )

(τ 2 − λ1 , u )

in the second equalit

y and (3. 1), (3. 2) in the second inequality also (3. 5) and (3. 7) in the third inequality. □
Repeating similar procedure in the sequel discussion we can get a generalized estimation for i t
h hitting times of trajectories when they hit A .
Theorem 3: let the i th hitting times of trajectories starting from x, z and evolving with fixed c
ontrol u be τ i , λi , and xi , zi be the points where the trajectories hit A .
Under the assumptions of Theorem 1, if x − z < δ i , then

τ i − λi < Ce L(τ ∨ λ ) P i −1 x − z
i

xi − zi < e

L (τ i ∨ λi )

(3. 8)

i

( FC + 1) Pi −1

x−z .

δ1e− L(τ ∨ λ )
i

i

(3. 9)

} and P = FC (1 + G ) + G .
P i −1
The proof is similar with theorem 2.
Corollary: Assume ( A1 )-( A9 ) to be satisfied. Let the reach times of trajectories starting from

Here δ i = min{δ1 , δ 2 , L ,

x, z and evolving with fixed control u be τ = t x ( u ) , λ = t z ( u )
Denote x = y x (τ , u ) , z = yz ( λ , u ) .
Then we get

8

L τ ∨λ
τ − λ < Ce ( ) P n x − z

(3. 10)

L τ ∨λ
x − z < e ( ) ( FC + 1) P n x − z

(3. 11)

for x − z < δ , where δ = min{δ1 , L , δ n ,

− L τ ∨λ
δ1e ( )

} and n is the number of times that the traj
Pn
ectories hit the jump set A before they arrive at a target set.
The proof goes through almost the same way as in Theorem 1.
However if we give δ defined above, then y ( λn − τ n , u ) − zn′ < δ1 and
xn′

⎛ β⎞
(where δ * is defined in Lemma), so y z ′ (τ − λn , u ) ∈ B ⎜ x , ⎟ .
n
n
2
⎝ 2⎠
From the lemma it is clear that if the trajectory starting from xn′ arrives at a target set then the t

x − yz ′ (τ − λn , u ) < δ * <

β

rajectory starting from zn′ also arrives at a target set without hitting Aj , which means the number o
f times of the trajectories starting from x, z respectively and evolving with u is equal if the initial p
oints are closer enough to be satisfied x − z < δ .
Now we deal with the continuity of the value function through the following theorem.
Theorem 4: Under the assumptions ( A1 )-( A12 ), the value function V defined in (2. 5) is bounde
d and if V is lower semicontinuous at each point of ∂Γ , then it is locally Holder continuous on
Ω \ Γ with respect to the initial point.
(Proof) First of all we show the value function is bounded.
From the definition of the value function we get for any u ( ⋅) ∈ u , and v ∈ V1
V ( x, q ) ≤

tx (u )

λτ
λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + τ ∑( ) C ( x , q , v ) e
−

− t

x

0

i <t x u

(

a

i

i

i

)

+ e − λtx ( u ) h y x ( t x ( u ) , u ) .
By assumptions ( A8 )-( A12 )
K
K
V ( x, q ) ≤ 0 + C0 ∑ e − λτ i + H ≤ 0 + C0 e − λτ1

λ

λ

τ i <t x ( u )

1
1− e

−

λβ

+ H , this proves the value function is

F

bounded.
Here we used h ( x ) is bounded by H on ∂Γ , that is h ( x ) < H

( x ∈ ∂Γ )

Now we prove the continuity of the value function defined in (2. 5).
Let choose x, z ∈ Ω \ Γ to be x − z < δ .
Considering the definition of the value function we assume that the controller chooses not to m
ake any controlled jump in C .
Then by the definition of V for any ε > 0 , there exist some control u ( ⋅) ∈ u , v ∈ V1 such that t
he following inequality holds:
V ( z, q ) ≥

tz (u )

λλ
λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + λ ∑( ) C ( z , q , v ) e
−

− t

z

0

(

i <t z

)

a

i

i

i

u

+ e − λtz ( u ) h yz ( t z ( u ) , u ) − ε ,
where {τ i }, {λi } are sequences of hitting times of the trajectories starting from x, z and evolvin

g with control u ( ⋅) ∈ u , v ∈ V1
Also

9

V ( x, q ) ≤

tx (u )

λτ
λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + τ ∑( ) C ( x , q , v ) e
0

+e

−

− t

x

− λt x ( u )

(

h yx ( t x ( u ) , u )

i <t x u

)

a

i

i

i

Let t x ( u ) = s%, t z ( u ) = t% .
If s% = t% = +∞ then (2. 5) is equal to the infinite horizon hybrid optimal control problem for wh
ich the continuity of the value function is proved in [1].
Assume s% ≤ t% < +∞ .
Define a new control u% ( t ) = u ( t + s% ) and denote the points of trajectories at time s% by
z% = y z ( s%, u ) , x% = y x ( s%, u ) ∈ ∂Γ

Then we have
s%

V ( z , q ) ≥ ∫ K ( yz ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( zi , qi , v ) e − λλi
λi < s%

0

t%

+ ∫ K ( y z ( t , u ) , q ( t ) , u ( t ) ) e − λt dt +
s%

C ( z , q , v ) e λλ + e λ h ( y ( t%, u ) ) − ε
∑
λ
−

s% ≤ i <t%

a

i

− t%

i

i

z

s%

= ∫ K ( yz ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( zi , qi , v ) e − λλi + e − λ s% J ( z%, q ( s% ) , u% , v )
λi < s%

0
s%

≥ ∫ K ( yz ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( zi , qi , v ) e − λλi + e − λ sV ( z%, q ( s% ) )
λi < s%

0

Hence

V ( x, q ) − V ( z , q ) ≤
s%

λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) − K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt
− t

x

z

0

∑

+

τ i , λi < s%

Ca ( zi , qi , v ) − Ca ( xi , qi , v ) e

(

)

+e − λ s% h ( x% ) − V ( z%, q ( s% ) ) + ε

(3. 12)

− λ (τ i ∨ λi )

( x% ∈ ∂Γ )

Let I = K ( y x ( t , u ) , q ( t ) , u ( t ) ) − K ( y z ( t , u ) , q ( t ) , u ( t ) ) and consider

s%

∫ Ie

− λt

dt .

0

If we split the interval [ 0, s% ] into a small intervals by τ i , λi so that where the system does not
make any jumps, then we can calculate

s%

∫ Ie

− λt

dt separately in three cases.

0

λi

Case 1:

∫ Ie
τ
i

− λt

τi

dt or

∫ Ie
λ

− λt

dt

i

In this case by using the Lipschitz continuity of K we have
λi

∫ Ie
τ

− λt

dt ≤ 2 K 0 τ i − λi

(3. 13)

i

τi

We can go through the same procedure for the calculation of

∫λ Ie
i

10

− λt

dt that is

τi

∫ Ie
λ

− λt

dt ≤ 2 K 0 τ i − λi

i

τ i +1

∫ Ie
λ

Case 2:

− λt

λi +1

∫ Ie
λ

dt or

i

− λt

dt

i

In this case it means that τ i < λi so using the Lipschitz continuity of K
τ i +1

∫
λ

Ie − λt dt =

τ i +1

i

∫
λ

(

) (

)

K y x ′ ( t − τ i , u ) , q ( t ) , u ( t ) − K y z ′ ( t − λi , u ) , q ( t ) , u ( t ) e− λt dt
i

i

,

i

τ i +1

≤ K1 ∫ y x ′ ( t − τ i , u ) − yz ′ ( t − λi , u ) e − λt dt
i

λi

i

In the integral of right side of above inequality
y x ′ ( t − τ i , u ) − yz ′ ( t − λi , u ) = y y
i

i

≤e

xi′

L ( t − λi )

( t − λi , u ) − yz ′ ( t − λi , u )

( λi −τ i , u )

i

yx ′ ( λi − τ i , u ) − zi′ ,
i

where we used the semigroup property yx ′ ( t − τ i , u ) = y y
i

ty and (3. 1) in the next inequality.
On the other hand
y ( λi − τ i , u ) − zi′ ≤ y
xi′

xi′

( λi − τ i , u ) − xi′

xi′

( λi −τ i , u )

( t − λi , u )

in the first equali

+ xi′ − zi′

and from (3. 2) and (3. 8)

yx ′ ( λi − τ i , u ) − xi′ ≤ F λi − τ i ≤ FCe Lλi P i −1 x − z ,
i

Also by using Lipschitz continuity of g and (3. 9)

xi′ − zi′ ≤ G xi − z i ≤ Ge Lλi ( FC + 1) P i −1 x − z .
Combining two inequalities we get
yx ′ ( λi − τ i , u ) − xi′ = P i e Lλi x − z
i

and

yx ′ ( t − τ i , u ) − yz ′ ( t − λi , u ) = P i e Lt x − z
i

i

Hence
τ i +1

∫ Ie
λ

− λt

dt = K1 P x − z

τ i +1

i

i

λi +1

Similarly for

∫λ Ie

− λt

i

dt we have

i

λi +1

∫ Ie
λ

− λt

i

τ i +1

Case 3:

∫ Ie
τ
i

− λt

λi +1

dt or

∫ Ie
τ

− λt

∫e
λ

dt = K1 P x − z
i

λi +1

∫e
λ
i

dt

i

11

( L − λ )t

dt .

( L − λ )t

dt .

(3. 14)

Likewise in case 2 it can be written
τ i +1

∫ Ie
τ

− λt

τ i +1

dt ≤ K1 ∫ y z ′ ( t − λi , u ) − y x ′ ( t − τ i , u ) e − λt dt
τi

i

1

i

τ i +1

≤ K1 ∫ y y
τi

z1′

(τ i − λi )

τ i +1

≤ K1 ∫ e

( t − τ i , u ) − yx ′ ( t − τ i , u ) e− λt dt
i

L ( t −τ i )

i Lτ i

Pe

− λt

x − z e dt ≤ K1 P x − z
i

τi

τ i +1

∫e
τ

( L − λ )t

dt

(3. 15)

i

Also in the same way we have
λi +1

∫ Ie
τ

− λt

dt = K1 P x − z

λi +1

∫e
τ

i

i

( L − λ )t

dt

i

Now we sum up all the integrals to calculate

s%

∫ Ie

− λt

dt .

0

Let the trajectories starting from initial points make n times of jumps on ∂A until it reaches a t
arget set.
Define τ 0 = 0, τ n +1 = s% , then using (3. 8), (3. 13), (3. 14), (3. 15)
s%

n

n

s%

o

i =1

i =0

o

( L − λ )t
− λt
i −1 Ls%
i
∫ Ie dt ≤ ∑ 2 K0CP e x − z + ∑ K1P x − z ∫ e dt
n
⎧
⎛ P n +1 − 1 ⎞ e( L −λ ) s% − 1
Ls% ⎛ P − 1 ⎞
x−z
⎪2 K 0Ce ⎜
⎟ x − z + K1 ⎜
⎟⋅
⎪
⎝ P −1 ⎠
⎝ P −1 ⎠ L − λ
=⎨
n
n +1
⎪ 2 K Ce Ls% ⎛ P − 1 ⎞ x − z + K ⎛ P − 1 ⎞ ⋅ s% x − z
⎜
⎟
⎟
1⎜
⎪ 0
⎝ P −1 ⎠
⎝ P −1 ⎠
⎩
On the other hand by Lipschitz continuity of Ca and (3. 9) we get
n

∑ Ca ( xi , qi , v ) − Ca ( zi , qi , v ) e

− λ (τ i ∨ λi )

i =1

n

≤ ∑ 2C1 xi − zi e

L≠λ
(3. 16)
L=λ

− λ (τ i ∨ λi )

i =1

⎛ Pn −1 ⎞
≤ 2C1 ∑ ( FC + 1) e Ls% P i −1 x − z ≤ 2C1 ( FC + 1) e Ls% ⎜
⎟ x−z
i =1
⎝ P −1 ⎠
We can think some constant a so that the following inequality holds
Pn −1
< aP n .
P −1
In fact it is possible if we give F , G, C large constant then P is also large enough to be satisfi
1
ed above inequality. (For example it is sufficient when we give a =
.)
P −1
%
β
β
sF
, then by (3. 12), (3. 16) it follows that
Also we have τ i +1 − τ i ≥ , s% ≥ τ n +1 − τ 1 ≥ n ⋅ , n ≤
F
F
β
n

12

%
sF
⎧
%
sF
Ls%
β
e
P
⎪
Ls%
β
−
+
⋅
aK
Ce
P
x
z
aK
P
x−z +
2
0
1
⎪
L−λ
⎪
⎪
e − λ s% h ( x% ) − V ( z%, q ( s% ) ) + ε
L≠λ
V ( x, q ) − V ( z , q ) ≤ ⎨
%
%
sF
sF
⎪
⎪ 2aK 0Ce Ls% P β x − z + aK1 P ⋅ P β ⋅ s% x − z
⎪
⎪
+ e − λ s% h ( x% ) − V ( z%, q ( s% ) ) + ε
L=λ
⎩

(

)

(

(3. 17)

)

⎧
%
sF
⎪ ⎛
%
Now we fix some θ ∈ ( 0, 1) and define δ = min ⎨δ , ⎜ e( L +1) s ⋅ P β
⎜
⎪ ⎝
⎩
−θ ln δ
Let T =
.
F
L + 1 + ln P

1
− ⎫
⎞ θ⎪
.
⎟⎟ ⎬
⎠ ⎪
⎭

β

Then for any initial points x, z ( x − z < δ ),
T=

−θ ln x − z
−θ ln δ
−θ ln δ
<
<
F
F
F
L + 1 + ln P L + ln P L + ln P

β

β

β

and from the fact
TF

e LT P β < x − z
⇔ LT +
⇔T <

TF

TF

we have e P β < x − z

−θ

Also
s% <

ln P < −θ ln x − z

β
−θ ln x − z
L+

LT

−θ

F

β

ln P

. (Here we assume L +

F

β

ln P > 0 )

−θ ln δ
F
L + 1 + ln p

β

⇔ s% ( L + 1) + s%
⇔ e(

L +1) s%

%
sF

F

β

ln P < −θ ln δ

P β < δ −θ

%
⎛ ( L +1) s% sF
⇔ δ < ⎜e
⋅P β
⎜
⎝

⎞
⎟⎟
⎠

−

1

θ

then s% < T by the definition of δ .
Therefore (2. 17) can be written as follows
⎧ M 1 x − z 1−θ + e − λ s% h ( x% ) − V ( z%, q ( s% ) ) + ε
⎪
V ( x, q ) − V ( z , q ) ≤ ⎨
1−θ
− λ s%
h ( x% ) − V ( z%, q ( s% ) ) + ε
⎪⎩ M 2 x − z + e
Here M 1 , M 2 are some constants.

(
(

13

)
)

L≠λ
L=λ

(3. 18)

∂Γ .

To prove the local Holder continuity of the value function we observe that it is continuous on

We can choose such small neighborhood of a target set that no jumps can make in any points of
the neighborhood.
Then for control u
t z% ( u )

λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + h ( y ( t ( u ) , u ) ) − h ( x% )

V ( z%, q ( s% ) ) − h ( x% ) ≤

− t

z%

z%

z%

0

We show that the first term in the right side of above inequality tends to zero as z% → x% .
Define T ( x ) = inf t x ( u ) then we pick u * ∈ u such that t z% ( u ) ≤ T ( z% ) + ε for any ε > 0 .
u∈u

Then

t z% ( u )

λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt ≤ K t ( u ) ≤ K (T ( z% ) + ε ) ,
− t

0 z%

z%

0

0

where K 0 is defined in assumption ( A11 )
The right side tends to zero as z% → x% and ε → 0+ thanks the continuity of T (see the propositio
n 1. 2 in chapter IV of [3]).
Also y z% ( t z% ( u ) , u ) − x% ≤ yz% ( t z% ( u ) , u ) − z% + z% − x% → 0 and this means

(

)

h y z% ( t z% ( u ) , u ) − h ( x% ) → 0
by the continuity of h on Γ .
As a result, we get lim sup V ( z%, q ( s% ) ) ≤ h ( x% ) , x% ∈ ∂Γ .
z% → x%

Thanks to the assumption that V is lower semicontinuous on ∂Γ , it is continuous on ∂Γ .
Therefore (3. 18) tends to zero as x → z , because x% − z% ≤ e s% x − z → 0 as x → z and by the c

ontinuity of the value function on ∂Γ it follows that h ( x% ) − V ( z%, q ( s% ) ) → 0 as x → z .
Now consider in the case s% > t% .
In this case from the dynamic programming theory (we prove it in the next section) we have
t%

%
V ( x, q ) ≤ ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( xi , qi , v ) e − λτ i + e − λt V ( yx ( t%, u ) , q ( t% ) )

τ i <t%

0

t%

∴ V ( x, q ) − V ( z, q ) ≤ ∫ K ( yx ( t , u ) , q ( t ) , u ( t ) ) − K ( yz ( t , u ) , q ( t ) , u ( t ) ) e− λt dt
0

∑
τ λ

+

i,

i < s%

Ca ( zi , qi , v ) − Ca ( xi , qi , v ) e

− λ (τ i ∨ λi )

+ e − λt V ( y x ( t%, u ) , q ( t% ) ) − h ( yz ( t%, u ) ) + ε
%

( y ( t%, u ) ∈ ∂Γ )
z

Taking throughout the same procedure as in the case s% ≤ t% , we can prove the continuity of the
□
value function on Ω under the assumption that it is lower semicontinuous on ∂Γ .
4. Dynamic Programming Principle and Hamilton-Jacobi variational inequality

First we show the following theorem which is called for Dynamic Programming Principle.
Theorem5: Let V ( x, q ) be the value function defined in (2. 5).
1) If τ 1 is the first hitting time that trajectory starting from the initial point x and evol
ving with control u hits A , then
τ1

V ( x, q ) = inf{∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + e − λτ1 MV ( x1 , q1 )} ,
u

0

14

(4. 1)

where MV ( x, q ) = inf{V ( g ( x, q, v ) , q ) + Ca ( x, q, v )} .
v∈V1

2) If ξ1 is the first time that the controller chooses to make a jump in C , then
ξ1

V ( x, q ) = inf{∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + e − λξ1 NV ( y (ξ1 ) , q (ξ1 ) )} ,
u

where NV ( x, q ) =

(4. 2)

0

inf {V ( x′, q′ ) + Cc ( x, q, x′, q′ )} .

( x′, q′)∈D× I

3) For any T > 0
V ( x) =

T ∧t x (u )

inf

∫

{

u , v , ξi , y (ξi )′

0

K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( xi , qi , v ) e− λτ i

(

τ i <T

)

+ ∑ Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e − λξi
ξi <T

(

)

− λ T ∧t u
+ e ( x ( ))V y x (T ∧ t x ( u ) , u ) , q (T ∧ t x ( u ) ) } ,

(4. 3)

where a ∧ b = min{a, b} .
(Proof) First let’s prove 1).
If τ 1 is the first hitting time of A , then
τ1

V ( x, q ) ≤ ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + Ca ( x1 , q1 , v ) e− λτ1
0

tx (u )

+(

λτ
λ
∫τ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + τ τ∑ ( ) C ( x , q , v ) e

∑
ξ
i <t x

(u)

1 < i <t x

(

1

+

−

− t

x

)

a

i

i

i

u

(

)

Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e− λξi + e− λtx ( u ) h yx ( t x ( u ) , u ) )

From the third term in the right side of above inequality we change the variable t ′ = t − τ 1 in the
bracket and after that taking infimum in the bracket over the control variables we have
τ1

V ( x, q ) ≤ ∫ K ( yx ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + e− λτ1 Ca ( x1 , q1 , v )
0

+e

V ( g ( x1 , q1 , v ) , q (τ 1 ) )

− λτ1

Here again taking the infimum in the last two terms over the discrete control v ∈ V1 and taking i
nfimum over the control u ∈ u again, then we get
τ1

V ( x, q ) ≤ inf{∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + e − λτ1 MV ( x1 , q1 )}
u

0

For the reverse inequality if we give any ε > 0 , then thanks to the definition of the value functi
⎛
′ ⎞
on there exist some control θ = ⎜ u , v , ξi , y ξi ⎟ such that
⎝
⎠

( )

τ1

(

)

V ( x, q ) ≥ ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + Ca x1 , q1 , v e− λτ1 +
0

tx (u )

(

λτ
λ
∫τ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + τ τ∑ ( ) C ( x , q , v ) e
1

+

−

− t

x

a

1 < i <t x

i

i

i

u

⎛
′
′⎞
− λt u
Cc ⎜ y ξi , q ξi , y ξi , q ξi ⎟ e − λξi + e x ( ) h y x ( t x ( u ) , u ) ) − ε
⎝
⎠
i <t x ( u )

∑
ξ

( ) ( ) ( ) ( )

(

15

)

We change the variable t ′ = t − τ 1 in the bracket and taking infimum over the control variables i
n the bracket, then we get
τ1

(

V ( x, q ) ≥ ∫ K ( yx ( t , u ) , q ( t ) , u ( t ) ) e− λt dt + e− λτ1 Ca x1 , q1 , v
0

( (

)

) ( ))

+e − λτ1V g x1 , q1 , v , q τ 1 − ε
Taking infimum over the discrete control v ∈ V1 in the last two terms and taking infimum again
over the control u ∈ u , then
τ1

V ( x ) ≥ inf{∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + e− λτ1 MV ( x1 , q1 )} + ε
u

0

Hence as ε → 0 we get the reverse inequality.
We can prove 2) in a similar way as in 1).
Now proceed to prove 3).
If T ≥ t x ( u ) then (4. 3) is equivalent to the definition of the value function (2. 5).
Let T < t x ( u ) . (4. 3) can be written as follows.
V ( x, q ) =

(

T

{∫ K ( yx ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( xi , qi , v ) e − λτ i

inf

u , v , ξi , y (ξi )′

0

τ i <T

)

+ ∑ Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e − λξi + e− λ (T )V ( yx (T , u ) , q (T ) )}
ξi <T

By the definition of the value function for any control u we have
T

V ( x, q ) ≤ ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( xi , qi , v ) e− λτ i
0

τ i <T

(

)

+ ∑ Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e − λξi
ξi <T

tx (u )

+(

λτ
λ
∫ K ( y ( t , u ) , q ( t ) , u ( t ) ) e dt + τ∑ ( ) C ( x , q , v ) e

T

∑
ξ

+

−

− t

x

T ≤ i <t x ( u )

T ≤ i <t x u

(

a

)

i

i

i

(

)

t u
Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e − λξi + e − λ x ( ) h yx ( t x ( u ) , u ) )

We change the variables t ′ = t − T in the bracket in the right side of above inequality and taking
infimum over the control variables. Hence
T

V ( x, q ) ≤ ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( xi , qi , v ) e− λτ i
0

τ i <T

(

)

+ ∑ Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi ) e − λξi + e − λT V ( y x (T , u ) , q (T ) )
ξi <T

Continue taking infimum over the control variables in the right side of above inequality to get
V ( x, q ) ≤

(

T

inf

{∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca ( xi , qi , v ) e − λτ i

u , v , ξi , y (ξi )′

0

τ i <T

)

+ ∑ Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e − λξi + e − λT V ( yx (T , u ) , q (T ) )} (4. 4)
ξi <T

To get a reverse inequality let give any ε > 0 .

16

⎛
′ ⎞
By the definition of the value function there exist some control θ = ⎜ u , v , ξi , y ξi ⎟ such tha
⎝
⎠

( )

(

T

)

V ( x, q ) ≥ ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e − λt dt + ∑ Ca xi , qi , v e− λτ 1
τ i <T

0

t

t (u )

x
⎛
′
′⎞
+ ∑ Cc ⎜ y ξi , q ξi , y ξi , q ξi ⎟ e − λξi + ( ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e− λt dt
ξ i <T
⎝
⎠
T

( ) ( ) ( ) ( )

+

)

Ca xi , qi , v e − λτ i +

− λt x ( u )

h yx ( t x ( u ) , u ) ) − ε

T ≤ i <t x ( u )

+e

(

∑
τ

(

)

⎛
′
′⎞
Cc ⎜ y ξi , q ξi , y ξi , q ξi ⎟ e − λξi
⎝
⎠
T ≤ i <t x ( u )

( ) ( ) ( ) ( )

∑
ξ

Changing the variable t ′ = t − T in the bracket from the third term to last one and taking infimu
m over the control variables, then we have

(

T

)

V ( x, q ) ≥ ∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e− λt dt + ∑ Ca xi , qi , v e− λτ i
τ i <T

0

( ( )

)

⎛
′
′⎞
+ ∑ Cc ⎜ y ξi , q ξi , y ξi , q ξi ⎟ e − λξi + e − λTV y x T , u , q (T ) − ε
⎝
⎠
ξi <T
Taking infimum over the control variables again we get the reverse inequality as ε → 0

( ) ( ) ( ) ( )

V ( x, q ) ≥

T

inf

{∫ K ( y x ( t , u ) , q ( t ) , u ( t ) ) e− λt dt + ∑ Ca ( xi , qi , v ) e− λτ i

u , v , ξi , y (ξi )′

(

0

)

τ i <T

+ ∑ Cc y (ξi ) , q (ξi ) , y (ξi )′ , q (ξi )′ e − λξi + e − λTV ( y x (T , u ) , q (T ) )}
ξi <T

(4. 5)

From (4. 4) and (4. 5) we prove 3).
□
Now we derive the Hamilton-Jacobi inequality that the value function must be satisfied by usin
g Dynamic Programming Principle.
The value function of the classical reach time problems which is controlled by only continuous
control usually is given as a viscosity solution of Dirichlet boundary problem with the terminal cos
t h as a boundary data.
In hybrid optimal control problem involving both of the continuous and discrete controls the p
artial differential equation satisfied by the value function is represented as a quasi-variational inequ
ality. Also the Hamiltonian is given as follows
⎧ − K ( x, q , u ) − f ( x, q , u ) ⋅ p ⎫
H ( x, q, p ) := sup ⎨
⎬
λ
u∈U ⎩
⎭
We prove the following theorem which gives a method to determine the value function as a vis
cosity solution of the Hamilton-Jacobi variational inequality.
Theorem6: Assume ( A1 )-( A12 ). If the value function of the hybrid optimal control problem wit
h reach time to a target set defined in (2. 5) is lower semicontinuous on ∂Γ , then it satisfies the foll
owing quasi-variational inequality in the viscosity sense.
⎧ V ( x, q ) − MV ( x, q ) = 0
( x, q ) ∈ A × I
⎪
⎪max{V ( x, q ) − NV ( x, q ) , V ( x, q ) + H ( x, q, D xV ( x, q ) )} = 0 ( x, q ) ∈ C × I
(4. 6)
⎨
( x, q ) ∈ Ω \ ( A ∪ C ∪ Γ ) × I
⎪V ( x, q ) + H ( x, q, DxV ( x, q ) ) = 0
⎪ V ( x, q ) − h ( x ) = 0
( x, q ) ∈ ∂Γ × { j}
⎩

17

Remark3. By subsolution (respectively, supersolution) of Dirichlet condition V ( x, q ) = h ( x, q )
on ( x, q ) ∈ ∂Γ × { j} we mean a function ≤ h (respectively, ≥ h ) at each point of ( x, q ) ∈ ∂Γ × { j}
The proof of the Theorem 6 is essentially the same as the proof of the Theorem 4. 2 of [1] and
we omit it.
5. Uniqueness

In this section we prove the comparison between any two viscosity solutions to demonstrate the
uniqueness of the viscosity solutions of QVI.
The following theorem is the extension of theorem 5. 1 of [1] to a comparison result for a Diric
hlet boundary value problem.
Theorem 7: Assume ( A1 )-( A12 ). If the value function defined in (2. 5) is continuous then it is t
he unique solution of (4. 6) in viscosity sense.
Remark4. The condition of continuity of the value function can be weakened in the way that it is
lower semicontinuous on ∂Γ . In this case V is the unique solution of QVI in the viscosity sense tha
nks to theTheorem4.
(Proof) The idea of proof is quite similar with the one noted in the proof of Theorem 5. 1 of [1]
so we briefly note here comparing with the proof of above Theorem.
Let u1 , u2 , bounded and continuous functions on Ω \ Γ , be two viscosity solutions of QVI (give
n by (4. 6))
Define the auxiliary function Φ on Ω × Ω as follows:
1
2
2
2
Φ q ( x, y ) = u1 ( x ) − u2 ( y ) − x − y − k x + y , ( x, y ) ∈ Ω q × Ω q

ε

(

)

where ε and k are small positive parameters to be chosen conveniently.
Now we prove sup sup Φ q ( x, y ) ≤ 0
q

Ω q ×Ω q

⎧ C C′ ⎫
Suppose that sup sup Φ q ( x, y ) = C > 0 and fix k such that k < min ⎨ , ⎬ where C ′ is the cons
q Ωq ×Ωq
⎩2 2 ⎭
tant to be chosen such that Ca , Cc are bounded below. (See assumption ( A12 ) .)
Then we can choose some point ( xk , yk ) ∈ Ω1 × Ω1 without loss of generality such that
C
by the definition of supremum.
2
Let Φ1 attain its supremum at some points ( x0 , y0 ) in Ω1 × Ω1 . We can choose this point because t
Φ1 ( xk , yk ) > C − k >

hanks to the structure of Φ and boundness of u1 , u2 , Φ1 ( x, y ) → −∞ as x , y → ∞ .
We first summarize the following estimates needed in the proof of the uniqueness.
Lemma2. The following statements are true.
(1)
(2)
(3)

x0 − y0

2

≤ D for some D independent of ε and k .

ε

k x0 , k y0 ≤ Dˆ for some D̂ independent of ε and k .
x0 − y0

ε

2

≤ ωk

(

Cε

) where ω

k

is the local modulus of continuity of both u1 , u2 in the ball

of radius R ( k ) , dependent on k , but independent of ε .
The proof is listed in [1].
If j ≠ 1 that is Γ ∈
/ Ω1 then all the statement in the proof of Theorem 5. 1 of [1] can be dropped
with no changes in the proof. If not we have some differences.
18

The only difference in the proof is one of the ( x0 , y0 ) may belong in Γ .
If ( x0 , y0 ) ∈ Γ × A or A× Γ it is straightforward that it is contradicted to (2) of Lemma2 becaus
e of the assumption ( A8 ) .
In the case of ( x0 , y0 ) ∈ Γ × C or C × Γ we can get a contradiction in the same way.
Now consider in the case ( x0 , y0 ) ∈ Γ × Γ or ( x0 , y0 ) ∈ Γ × Ω \ ( A U C U Γ ) or

( x0 , y0 ) ∈ Ω \ ( A U C U Γ ) × Γ .

Without loss of generality let x0 ∈ ∂Γ .
u1 and u2 are respectively the viscosity subsolution and supersolution, so we get u1 ≤ h ≤ u2 on
∂Γ . Then
1
2
2
2
Φ1 ( x0 , y0 ) = u1 ( x0 ) − u2 ( y0 ) − x0 − y0 − k x0 + y0

(

ε

)

≤ u1 ( x0 ) − u2 ( x0 ) + u2 ( x0 ) − u2 ( y0 ) ≤ u2 ( x0 ) − u2 ( y0 ) ≤ ω

(

Cε

)

where ω is the modulus of continuity of u2 in the ball of radius R ( k ) , dependent on k , but indepe
ndent of ε .
By choosing ε small enough we get a contradiction to sup sup Φ q ( x, y ) = C > 0 .
q

Ω q ×Ω q

If x0 ∈ ∂Γ we add and subtract u1 ( y0 ) and use the modulus of continuity of u1 to conclude in
the same way.
The other part of the proof is the same as in the proof of Theorem 5. 1 of [1].

□

References
[1] S. Dharmatti and M. Ramaswamy. Hybrid control systems and viscosity solutions.
SIAM J. Control Optim. 44:1259-1288, 2005.
[2] G. Barles, S. Dharmatti and M. Ramaswamy. Unbounded viscosity solutions of hybrid
control systems. ArXiv preprint, 2008.
[3] M. Bardi and I. Capuzzo-Dolcetta. Optimal Control and Viscosity Solutions of HamiltonJacobi- Bellman Equations. Birkhauser, Boston, 1997
[4] C.J. Tomlin, G.J. Pappas, and S.S. Sastry. Conflict resolution for air traﬃc management:
A case study in multi-agent hybrid systems. IEEE Transactions on Automatic Control,
43(4):509–521, 1998.
[5] M. S. Branicky, V. S. Borkar, and S. K. Mitter, A unified framework for hybrid control
problem, IEEE Trans. Automat. Control, 43 (1998), pp. 31–45.
[6] A. Balluchi, L. Benvenuti, M.D. Di Benedetto, C. Pinello, and A.L. Sangiovanni-incentelli.
Automotive engine control and hybrid systems: Challenges and opportunities.
Proceedings of the IEEE, 88(7):888–912, July 2000.
[7] S. Engell, S. Kowalewski, C. Schulz, and O. Stursberg. Continuous-discrete interactions in
chemical processing plants. Proceedings of the IEEE, 88(7):1050–1068, July 2000.
[8] Myong Song Ho, Kwang Nam O. On the value function of the hybrid optimal control
problem with reach time to a target set. In preparation.

19

