Simultaneous State and Parameter Estimation for Second-Order Nonlinear
Systems

arXiv:1703.07068v1 [] 21 Mar 2017

Rushikesh Kamalapurkar
Abstract— In this paper, a concurrent learning based adaptive observer is developed for a class of second-order nonlinear
time-invariant systems with uncertain dynamics. The developed
technique results in uniformly ultimately bounded state and
parameter estimation errors. As opposed to persistent excitation
which is required for parameter convergence in traditional
adaptive control methods, the developed technique only requires
excitation over a finite time interval to achieve parameter
convergence. Simulation results in both noise-free and noisy
environments are presented to validate the design.

I. I NTRODUCTION
Owing to Increasing reliance on automation and increasing
complexity of autonomous systems, the ability to adapt
has become an indispensable feature of modern control
systems. Traditional adaptive control methods (see, e.g.,
[1]–[3]) attempt to improve the tracking performance, and
in general, do not focus on parameter estimation. While
accurate parameter estimation can improve robustness and
transient performance of adaptive controllers, (see, e.g., [4]–
[6]), parameter convergence typically requires restrictive
assumptions such as persistence of excitation. An excitation
signal is often added to the controller to ensure persistence of
excitation; however, the added signal can cause mechanical
fatigue and compromise the tracking performance.
Parameter convergence can be achieved under a finite
excitation condition using data-driven methods such as concurrent learning (see, e.g., [6]–[8]), where the parameters
are estimated by storing data during time-intervals when
the system is excited, and then utilizing the stored data to
drive adaptation when excitation is unavailable. Concurrent
learning has been shown to be an effective tool for adaptive
control (see, e.g., [6]–[9]) and adaptive estimation (see, e.g.,
[10]–[15]), however, concurrent learning typically requires
full state feedback along with accurate numerical estimates
of the state-derivative.
Novel concurrent learning techniques that can be implemented using full state measurements but without numerical
estimates of the state-derivative are developed in [16] and
[17]; however, since full state feedback is typically not
available, the development of an output-feedback concurrent
learning framework is well-motivated. An output feedback
concurrent learning technique is developed for second-order
linear systems in [18]; however, the implementation critically
depends on the certainty equivalence principle, and hence, is
not directly transferable to nonlinear systems.
Rushikesh Kamalapurkar is with the School of Mechanical and
Aerospace Engineering, Oklahoma State University, Stillwater, OK, USA.
rushikesh.kamalapurkar@okstate.edu.

In this paper, an output feedback concurrent learning
method is developed for simultaneous state and parameter
estimation in second-order uncertain nonlinear systems. An
adaptive state-observer is utilized to generate estimates of the
state from input-output data. The estimated state trajectories
along with the known inputs are then utilized in a novel datadriven parameter estimation scheme to achieve simultaneous
state and parameter estimation. Convergence of the state estimates and the parameter estimates to a small neighborhood
of the origin is established under a finite (as opposed to
persistent) excitation condition.
The paper is organized as follows. An integral error system
that facilitates parameter estimation is developed in Section
II. Section III is dedicated to the design of a robust state
observer. Section IV details the developed parameter estimator. Section V details the algorithm for selection and storage
of the data that is used to implement concurrent learning.
Section VI is dedicated to a Lyapunov-based analysis of the
developed technique. Section VII demonstrates the efficacy
of the developed method via a numerical simulation and
Section VIII concludes the paper.
II. E RROR S YSTEM FOR E STIMATION
Consider a second order nonlinear system of the form1
ṗ (t) = q (t) ,
q̇ (t) = f (x (t) , u (t)) ,
y (t) = p (t) ,

(1)

where p : R≥T0 → Rn and q : R≥T0 → Rn denote
the generalized position states
generalized velocity
 T andT the
T
q
states, respectively, x , p
is the system state,
f : Rn × Rm → Rn is locally Lipschitz continuous, and
y : R≥T0 → Rn denotes the output. The model f is
comprised of a known nominal part and an unknown part,
i.e., f = f o + g, where f o : Rn × Rm → Rn is known
and locally Lipschitz and g : Rn × Rm → Rn is unknown
and locally Lipschitz. The objective is to design an adaptive
estimator to identify the unknown function g, online, using
input-output measurements. It is assumed that the system
is controlled using a stabilizing input, i.e., x, u ∈ L∞ . It
is further assumed that the signal p, and u are available
for feedback. Systems of the form (1) encompass secondorder linear systems and Euler-Lagrange models, and hence,
represent a wide class of physical plants, including but not
1 For a ∈ R, the notation R
≥a denotes the interval [a, ∞) and the
notation R>a denotes the interval (a, ∞).

limited to robotic manipulators and autonomous ground,
aerial, and underwater vehicles.
Given a compact set χ ⊂ Rn × Rm , and a constant
, the unknown function g can be approximated using
basis functions as g (x, u) = θT σ (x, u) +  (x, u), where
σ : Rn × Rm → Rp and  : Rn × Rm → Rn denote
the basis vector and the approximation error, respectively,
θ ∈ Rp×n is a constant matrix of unknown parameters,
and there exist σ, θ > 0 such that sup(x,u)∈χ σ (x, u) <
σ, sup(x,u)∈χ ∇σ (x, u) < σ, sup(x,u)∈χ  (x, u) < ,
sup(x,u)∈χ ∇ (x, u) < , and kθk < θ. To obtain an
error signal for parameter identification, the system in (1)
is expressed in the form
q̈ (t) = f o (x (t) , u (t))+θT σ (x (t) , u (t))+ (x (t) , u (t)) .
(2)
Integrating (2) over the interval [t − τ1 , t] for some constant
τ1 ∈ R>0 and then over the interval [t − τ2 , t] for some
constant τ2 ∈ R>0 ,

III. V ELOCITY ESTIMATOR DESIGN
To generate estimates of the generalized velocity, a velocity estimator inspired by [19] is developed. The estimator is
given by
p̂˙ = q̂
q̂˙ = f o (x̂, u) + θ̂T σ (x̂, u) + ν,

where x̂, p̂, q̂, and θ̂ are estimates of x, p, q, and θ,
respectively, and ν is a feedback term designed in the
following.
To facilitate the design of ν, let p̃ = p − p̂, q̃ = q − q̂,
θ̃ = θ − θ̂, and let
r (t) = p̃˙ (t) + αp̃ (t) + η (t) ,

o

T

(q (λ) − q (λ − τ1 )) dλ = If (t) + θ Iσ (t) + I (t) ,
t−τ2

(3)
denotes the integral operator f
7→
f (x (τ ) , u (τ )) dτ dλ. Using the Fundamental
t−τ2 λ−τ1
Theorem of Calculus and the fact that q (t) = ṗ (t), the
expression in (4) can be rearranged to form the affine
system
(4)

where

ν (t) = α2 p̃ (t) − (k + α + β) η (t) .

(
Iσ (t) , t ∈ [T0 +τ1 +τ2 , ∞) ,
G (t) ,
0
t < T 0 + τ1 + τ2 ,

(7)

and
(
E (t) ,

I (t) , t ∈ [T0 + τ1 + τ2 , ∞) ,
0
t < T0 + τ1 + τ2 .

(8)

The affine relationship in (4) is valid for all t ∈ R≥T0 ;
however, it provides useful information about the vector θ
only after t ≥ T0 + τ1 + τ2 .
The knowledge of the generalized velocity, q, is required
to compute the matrices F and G. In the following, a robust
adaptive velocity estimator is developed to generate estimates
of the generalized velocity.

(11)

(12)

The design of the signals η and ν to estimate the state from
output measurements is inspired by the p−filter [20]. Using
the fact that p̃ (T0 ) = 0, the signal η can be implemented
via the integral form
ˆt
η (t) = −

ˆt
(β + k) η (τ ) dτ −

T0



 p (t−τ2 −τ1 )−p (t−τ1 ) t ∈ [T +τ +τ , ∞) ,
0
1
2
P (t) ,
+p (t)−p (t−τ2 ) ,

0
t < T0 + τ1 + τ2 .
(5)
(
o
If (t) , t ∈ [T0 + τ1 + τ2 , ∞) ,
F (t) ,
(6)
0,
t < T0 + τ1 + τ2 ,

η (T0 ) = 0,

where α, k, and β are positive constants and the feedback
component ν is designed as

where
´t ´λI

P (t) = F (t) + θT G (t) + E (t) , ∀t ∈ R≥T0

(10)

where the signal η is added to compensate for the fact that
the generalized velocity state, q, is not measurable. Based on
the subsequent stability analysis, the signal η is designed as
the output of the dynamic filter
η̇ (t) = −βη (t) − kr (t) − αq̃ (t) ,

ˆt

(9)

kαp̃ (τ ) dτ −(k + α) p̃ (t) .

T0

(13)
The affine error system in (4) motivates the adaptive
estimation scheme that follows. The design is inspired by
the concurrent learning technique [21]. Concurrent learning
enables parameter convergence in adaptive control by using
stored data to update the parameter estimates. Traditionally,
adaptive control methods guarantee parameter convergence
only if the appropriate PE conditions are met [1, Chapter
4]. Concurrent learning uses stored data to soften the PE
condition to an excitation condition over a finite timeinterval. Concurrent learning methods such as [6] and [8]
require numerical differentiation of the system state, and
concurrent learning techniques such as [17] and [16] require
full state measurements. In the following, a concurrent
learning method that utilizes only the output measurements
is developed.
IV. PARAMETER E STIMATOR D ESIGN
To obtain output-feedback concurrent learning update law
for the parameter estimates, a history stack, denoted by
H,
n is utilized.
oThe history stack is a set of ordered pairs
M

Pi , F̂i , Ĝi

i=1

such that

Pi = F̂i + θT Ĝi + Ei , ∀i ∈ {1, · · · , M } ,

(14)

where Ei is a constant matrix. If a history stack that satisfies
(14) is not available a priori, it is recorded online, based
on the relationship in (4), by selecting an increasing set of
M
time-instances {ti }i=1 and letting

denote
the
operator
f
7→
f
(x̂
(τ
)
,
u
(τ
))
dτ
dλ.
In
this
case,
the
error
t−τ2 λ−τ1
term
by Ei = E (ti ) + F (ti ) − F̂ (ti ) +
 Ei is given 
θT G (ti ) − Ĝ (ti ) . Let [t1 , t2 ) be an interval over which
the history stack was recorded. Provided the states and
the state estimates remain within a compact set χ over
I , [t1 − τ1 − τ2 , t2 ), the error terms can be bounded as

error bound introduced by ) represent the system dynamics.
Hence, the history stack needs to be purged whenever better
estimates of the state are available.
Since the state estimator exponentially drives the estimation error to a small neighborhood of the origin, a newer
estimate of the state can be assumed to be at least as good
as an older estimate. A dwell time based greedy purging
algorithm is developed in this paper to utilize newer data for
estimation while preserving stability of the estimator.
The algorithm maintains two history stacks, a main history stack and a transient history stack, labeled H and G,
respectively. As soon as the transient history stack is full
and sufficient dwell time has passed, the main history stack
is emptied and the transient history stack is copied into the
main history stack. The sufficient dwell time, denoted by T ,
is determined using a Lyapunov-based stability analysis.
Parameter identification in the developed framework imposes the following requirement on the history stack H.
n
oM
Definition 1. A history stack
Pi , F̂i , Ĝi
is called
i=1
full rank if there exists a constant c ∈ R such that

kEi k ≤ L1  + L2 x̃I , ∀i ∈ {1, · · · , M } ,

0 < c < λmin {G } ,

Pi = P (ti ) ,

F̂i = F̂ (ti ) ,

Ĝi = Ĝ (ti ) ,

(15)

where
(

Îf o (t) , t ∈ [T0 + τ1 + τ2 , ∞) ,
0,
t < T0 + τ1 + τ2 ,
(
Îσ (t) t ∈ [T0 +τ1 +τ2 , ∞) ,
Ĝ (t) ,
0
t < T0 + τ1 + τ2 ,

F̂ (t) ,

(16)

(17)

´where
´ λ Î
t

(18)

where x̃I , maxi∈{1,··· ,M } supt∈I kx̃ (t)k and L1 , L2 > 0
are constants.
The concurrent learning update law to estimate the unknown parameters is designed as
M

T
X
˙
Ĝi Pi − F̂i − θ̂T (t) Ĝi ,
θ̂ (t) = kθ Γ (t)

(19)

i=1

where kθ ∈ R>0 is a constant adaptation gain and Γ :
2
2
R≥0 → R(2n +mn)×(2n +mn) is the least-squares gain
updated using the update law
Γ̇ (t) = β1 Γ (t) − kθ Γ (t) G Γ (t) .

(20)

PM
where the matrix G ∈ Rp×p is defined as G , i=1 Ĝi ĜTi .
Using arguments similar 
to [1, Corollary 4.3.2], it can be
shown that provided λmin Γ−1 (T0 ) > 0, the least squares
gain matrix satisfies
Γ Ip ≤ Γ (t) ≤ Γ Ip ,

(21)

where Γ and Γ are positive constants, and In denotes an
n × n identity matrix.
V. P URGING
The update law in (19) is motivated by the fact that if the
full state were available for feedback and if the approximaT
tion error, , were zero, then using P1 · · · Pn
=

T

T
F1 · · · Fn
+ G1 · · · Gn
θ, the parameters could be estimated
via the

  least squares Testi−1
G1 · · · Gn
P1 · · · P n
mate θ̂LS = G
−


T
F1 · · · Fn
G −1 G1 · · · Gn
. However, since
the history stack contains the estimated terms F̂ and Ĝ,
during the transient period where the state estimation error
is large, the history stack does not accurately (within the

(22)

where λmin (·) denotes the minimum singular value of a
matrix.
Assumption 1. For a given M ∈ N and c ∈ R>0 , there
M
exists a set of time instances {ti }i=1 such that a history
stack recorded using (15) is full rank.
A singular value maximization algorithm is used
 to select

M
the time instances {ti }i=1 . That is, a data-point Pj , F̂j , Ĝj
in the history

 stack is replaced with a new data-point
P ∗ , F̂ ∗ , Ĝ∗ , where F̂ ∗ = F̂ (t), P ∗ = P (t), and Ĝ∗ =
Ĝ (t), for some t, only if
P



T
∗ ∗T
smin
Ĝ
Ĝ
+
Ĝ
Ĝ
X
i
i
i6=j
smin  Ĝi ĜTi + Ĝj ĜTj <
,
(1 + ζ)
i6=j

(23)
where smin (·) denotes the minimum singular value of a
matrix and ζ is a constant. To simplify the analysis, new
data points are assumed to be collected τ1 + τ2 seconds after
a purging event. Since the history stack is updated using a
singular value maximization algorithm, the matrix G is a
piece-wise constant function of time. The use of singular
value maximization to update the history stack implies that
once the matrix G satisfies (22), at some t = T , and for
some c, the condition c < λmin (G (t)) holds for all t ≥ T .
The developed purging method is summarized in Fig. 1.
A Lyapunov-based analysis of the parameter and the state
estimation errors is presented in the following section.
VI. S TABILITY A NALYSIS
Each purging event represents a discontinuous change in
the system dynamics; hence, the resulting closed-loop system
is a switched system. To facilitate the analysis of the switched
system, let ρ : R≥0 → N denote a switching signal such

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:

δ (T0 ) ← 0, η (T0 ) ← 0
if t > δ (t) + τ1 + τ2 and a data point is available then
if G is not full then
add the data point to G
else
add the data point to G if (23) holds
end if
if smin (G ) ≥ ξη (t) then
if t − δ (t) ≥ T (t) then
H ← G and G ← 0 . purge and replace H
δ (t) ← t
if η (t) < smin (G ) then
η (t) ← smin (G )
end if
end if
end if
end if

Fig. 1. Algorithm for history stack purging with dwell time. At each time
instance t, δ (t) stores the last time instance H was purged, η (t) stores the
highest minimum singular value of G encountered so far, T (t) denotes the
dwell time, and ξ ∈ (0, 1] denotes a threshold fraction.

that ρ (0) = 1, and ρ (t) = i + 1, where i denotes the
number of times the update H ← G was carried out over
the time interval (0, t). For some s ∈ N, let Hs denotes the
history stack active during
nthe time interval
o {t | ρ (t) = s}),
containing the elements
Psi , F̂si , Ĝsi
, and let
i=1,··· ,M

T
Esi
be theP
corresponding error term.P
To simplify the notation,
M
M
T
let Gs , i=1 Ĝsi ĜTsi , and Qs = i=1 Ĝsi Esi
.
Using (14) and (19), the dynamics of the parameter
estimation error can be expressed as

˙
θ̃ (t) = −kθ Γ (t) Gs (t) θ̃ (t) − kθ Γ (t) Qs (t) .

(24)

Since the functions Gs : R≥T0 → Rp×p and Qs : R≥T0 →
Rp×n are piece-wise continuous, the trajectories of (24), and
of all the subsequent error systems involving Gs and and Qs ,
are defined in the sense of Carathéodory. Algorithm 1 ensures
that there exists a constant g > 0 such that λmin {Gs } ≥
g, ∀s ∈ N.
Using the dynamics in (1), (9) - (11), and the design of
the feedback component in (12), the time-derivative of the
error signal r is given by
ṙ (t) = −kr (t)+f˜o (x, u, x̂)+θT σ̃ (x, u, x̂)−θ̃T σ̃ (x, u, x̂)

That is, for a given switching index s, Ts denotes the time
instance when the (s + 1)th subsystem is switched on. The
analysis is carried out separately over the time intervals
[Ts−1 , Ts ), s ∈ N, where T1 = T0 + τ1 + τ2 + tM . Since the
history stack H is not updated over the intervals [Ts−1 , Ts ),
s ∈ N, the matrices Gs and Qs are constant over each
individual interval. The history stack that is active over the
interval [Ts , Ts+1 ) is denoted by Hs . To ensure boundedness
of the trajectories in the interval t ∈ [T0 , T1 ), the history
stack H1 is arbitrarily selected to be full rank. The analysis is
carried out over
using the state
 the aforementionedintervals
T T
∈ R3n+np and
vectors Z , p̃T rT η T vec θ̃

T
Y , p̃T rT η T ∈ R3n as follows.
Interval 1: First, it is established that Z
is  bounded over [T0 , T1 ),  where the bound is
PM
O kZ (T0 )k +
i=1 E1i +  . Given some ε > 0,
the bound on Z is utilized to select gains such that
kY (T1 )k < ε.
Interval 2: The history stack H2 , which is active over
[T1 , T2 ), is recorded over [T0 , T1 ). Without loss of generality,
it is assumed that H2 represents the system better than
PM
H1 (which is arbitrarily selected), that is,
≥
i=1 E1i
PM
i=1 E2i . The bound on Z over [T1 , T2 ) is then shown
to be smaller than that over [T0 , T1 ), which utilized to show
that kY (t)k ≤ ε, for all t ∈ [T1 , T2 ).
Interval 3: Using (18), the errors E3i are shown to be
O (kY3i k + ) where Y3i denotes the
 value of Y at the
time when the point P3i , F̂3i , Ĝ3i was recorded. Using
the facts that the history stack H3 , which is active over
[T2 , T3 ), is recorded over [T1 , T2 ) and kY (t)k ≤ ε, for all
PM
t ∈ [T1 , T2 ), the error
i=1 E3i is shown to be O (ε + ).
If T3 = ∞ then it is established that lim supt→∞ kZ (t)k =
O (ε + ). If T3 < ∞ then the fact that the bound on Z
over [T2 , T3 ) is smaller than that over [T1 , T2 ) is utilized to
show that kY (t)k ≤ ε, for all t ∈ [T2 , T3 ). The analysis
is then continued in an inductive argument to show that
lim supt→∞ kZ (t)k = O (ε + ) and kY (t)k ≤ ε, for all
t ∈ [T2 , ∞).
The stability result is summarized in the following theorem.

+ θ̃T σ (x, u) +  (x, u) − α2 p̃ + (k + α) η, (25)
where σ̃ (x, u, x̂) = σ (x, u) − σ (x̂, u) and f˜o (x, u, x̂) =
f (x, u) − f (x̂, u). Since (x, u) 7→ f (x, u) and (x, u) 7→
σ (x, u) are locally Lipschitz, and since t 7→ u (t) is bounded,
given a compact set χ̂ ⊂ Rn × Rm × Rn , there exist
Lf , Lσ > 0 such that sup(x,u,x̂)∈χ̂ f˜o (x, u, x̂) ≤ Lf kx̃k
and sup(x,u,x̂)∈χ̂ kσ̃ (x, u, x̂)k ≤ Lσ kx̃k.
To facilitate the analysis, let {Ts ∈ R≥0 | s ∈ N}
be a set of switching time instances defined as Ts =
{t | ρ (τ ) < s + 1, ∀τ ∈ [0, t) ∧ ρ (τ ) ≥ s + 1, ∀τ ∈ [t, ∞)} .

Theorem 1. Let ε > 0 be given. Let the history stacks
H and G be populated using the algorithm detailed in
Fig. 1. Let the learning gains be selected to satisfy the
sufficient gain conditions in (28), (29), (34), and (38). Let
T ∈ R>0 be a time instance such that the system states
are exciting over [T0 , T ], that is, the history stack can be
replenished if purged at any time t ∈ [T0 , T ]. Assume that
over each switching interval {t | ρ (t) = s}, the dwell-time,
T , is selected such that T (t) = Ts , where Ts is selected to
be large enough to satisfy (37). Furthermore assume that the

excitation interval is large enough so that T2 < T .2 Then,
lim supt→∞ kZ (t)k = O (ε + ).
Proof. Provided H1 is full rank, then the candidate Lyapunov
function


2V (Z, t) , α2 p̃T p̃ + rT r + η T η + tr θ̃T Γ−1 (t) θ̃ (26)
can be utilized to establish boundedness of trajectories over
[Ts−1 , Ts ). The candidate Lyapunov function satisfies
2

2

v kZk ≤ V (Z, t) ≤ v kZk ,
(27)


1
1
2 1
2 1
where v , 2 max 1, α , /Γ and v , 2 min 1, α , /Γ .
The time-derivative of V along the trajectories of (10), (11),
(20), (24), and (25) is given by
 
1 
V̇ = −α3 p̃T p̃ − krT r − βη T η − tr θ̃T kθ G1 + β1 Γ−1 θ̃
2


T ˜o
T T
T T
+r f + r θ σ̃ + r θ̃ σ̂ + rT  − kθ tr θ̃T Qs .
Using the Cauchy-Schwartz inequality, the derivative can be
bounded as
2
1
2
2
2
V̇ ≤ −α3 kp̃k − k krk − β kηk − a θ̃ + Lf krk kx̃k
2
+ krk θLσ kx̃k + krk θ̃ kσ̂k + krk  + kθ θ̃ Qs ,
where a = kθ g + βΓ1 , and Qs is a positive constant such that
Qs ≥ kQs k. Provided


 12σ 2
,
k > max 2 (4 + α) Lf + θLσ ,
a

α3 > (1 + α) Lf + θLσ ,

(28)
β > Lf + θLσ ,
Young’s inequality and nonlinear damping can be used to
conclude that
2
α3
k
β
a
2
2
2
V̇ ≤ − kp̃k − krk − kηk −
θ̃
4 
2
6
 2
3kθ2 2
k 3Lσ
2
2
2
−
kx̃k krk +
+
Q ,
−
8
2a
k
2a s

2
2
Since kx̃k ≤ (1 + α) kZk , V̇ ≤ −ν kZk − ινs , in the
domain
s
(
)
ka
3n+np
D, Z∈R
| kZk <
.
12Lσ (1 + α)
p
That is, V̇ is negative definite on D provided kZk > ινs >

2
3k2 2
0, where v , 12 min α3 , k/2, β, a/3 and ιs , k + 2aθ Qs .
Theorem 4.18 from [22] can then be invoked to conclude
that provided


15Lσ (1 + α)
vι1
k>
max V s ,
,
(29)
av
v
where V s ≥ kV (Z (Ts−1 ) , Ts−1 )k is a constant, then V̇ ≤
− vv V + ιs , ∀t ∈ [Ts−1 , Ts ).
2 A minimum of two purges are required to remove the randomly
initialized data, and the data recorded during transient phase of the derivative
estimator from the history stack.

In particular, ∀t ∈ [T0 , T1 ) ,


v
v
v
V (Z (t) , t) ≤ V 1 − ι1 e− v (t−T0 ) + ι1 ,
v
v

(30)

where V 1 > 0 is a constant such that |V (Z (T0 ) , T0 )| ≤ V 1 .
Hence, ∀t ∈ [T0 , T1 ) ,
(q
)
r
r
1
v
θ̃ (t) ≤ θ1 ,
max
V 1,
ι1 .
(31)
v
v
If it were possible to use the inequality in (30) to conclude
that over [T0 , T1 ), V (Z (t) , t) ≤ V (Z (T0 ) , T0 ), then an
inductive argument could be used to show that the trajectories
decay to a neighborhood of the origin. However, unless
the history stack can be selected to have arbitrarily large
minimum singular value (which is generally not possible),
the constant vv ι1 cannot be made arbitrarily small using the
learning gains.
Since ιs depends on Qs , it can be made smaller by
reducing the estimation errors and thereby reducing the errors
associated with the data stored in the history stack. To that
end, consider the candidate Lyapunov function
α2 T
1
1
p̃ p̃ + rT r + η T η.
2
2
2
The candidate Lyapunov function satisfies
W (Y ) ,

2

(32)

2

w kY k ≤ W (Y, t) ≤ w kY k ,
(33)


where w , 12 max 1, α2 , w , 12 min 1, α2 . In the
interval [Ts−1 , Ts ), the time-derivative of W is given by


 
Ẇ = −α3 p̃T p̃ − krT r − βη T η + rT f˜o + θT − θ̃T σ̃


+ rT θ̃T σ + 
Using the Cauchy-Schwartz inequality, the derivative Ẇ can
be bounded as
2

2

2

Ẇ = − α3 kp̃k − k krk − β kηk
 
+ Lf + θ + θs Lσ krk kx̃k + (θs σ + ) krk ,
where θs > 0 is a constant such that θs ≥
supt∈[Ts−1 ,Ts ) θ̃ (t) .
Consider the time interval [T0 , T1 ). Provided
 
k ≥ 1 + θ12 + Lf + θ + θ1 Lσ (4 + α)
 
α3 ≥ (1 + α) Lf + θ + θ1 Lσ
 
(34)
β ≥ Lf + θ + θ1 Lσ

1
3
then Ẇ ≤ − w
w W + λ, where w = 2 min α , k, β and
σ 2 +2
λ = 2 . That is, for all t ∈ [T0 , T1 ) ,


w
w
w
W (Y (t) , t) ≤ W 1 − λ e− w (t−T0 ) + λ,
(35)
w
w
where W 1 > 0 is a constant such that |W (Y (T0 ))| ≤ W 1 .
In particular, ∀t ∈ [T0 , T1 ) .
s


1
w
kY (t)k ≤
max W 1 , λ , kY k1 .
(36)
w
w

Provided the dwell time Ts is large enough so that


w
w
w
W s − λ e− w Ts ≤ λ,
w
w


v
v
v
V s − ιs e− v Ts ≤ ιs ,
v
v
then from (30) and (35), W (Y (T1 ))

≤

2wλ
wq

TABLE I
S IMULATION PARAMETERS FOR THE DIFFERENT SIMULATION RUNS .
T HE PARAMETERS ARE SELECTED USING TRIAL AND ERROR .
Noise Variance

(37)
and

2wλ
1
V (Z (T1 ) , T1 ) ≤ 2vι
. In particular, kY (T1 )k ≤
ww
q v
2vι1
and kZ (T1 )k ≤
vv . Note that the bound on Y (T1 ) can
be made arbitrarily small by increasing k, α, and β.
Now the interval [T1 , T2 ) is considered. Since the history
stack H2 which is active during [T1 , T2 ) is recorded during
[T0, T1 ), the bound
in (18) can be used to show that Q2 =

O kY k1 +  .

Since H1 is independent of the system trajectories, Q1
can be selected such that Q2 < Q1 , and hence, ι2 < ι1 .
Thus, provided the constant V 1 (and as a result, the gain k)
is selected large enough so that
2vι1
< V 1,
v

and hence, ∀t ∈ [T1 , T2 ),
s
2wλ
, kY k2 .
kY (t)k ≤
ww

0.001

T1
T2
N
Γ (t0 )
β1
α
k
β
ζ
ξ
kθ

0.5
0.3
50
I4
0.5
2
10
2
0
0.95
0.5/N

0.9
0.5
150
I4
0.5
2
10
2
0
0.95
0.5/N

10
8
6
4

The sufficient condition in (38) implies that V 2 < V 1 and
hence, (31) and ι2 < ι1 imply that θ2 < θ1 .
Since θ2 < θ1 , the gain conditions in (34) hold over
the interval [T1 , T2 ). A Lyapunov-based
analysis similar to
q

1
(32)-(36) yields kY (t)k ≤ w max W 2 , w
w λ . From (37),
2wλ
w ,

0

(38)

the gain condition in (29) holds over [T1 , T2 ), and hence, a
similar Lyapunov-based analysis, along with the bound V 2 =
2vι1
v can be utilized to conclude that ∀t ∈ [T1 , T2 ),
r
√
√
v
2ι1 , ι2 , θ2 .
(39)
max
θ̃ (t) ≤
vv

W2 =

Parameter

(40)

Now, the interval [T2 , T3 ) is considered. Since the history
stack H3 which is active during [T2 , T3 ) is recorded during
[T1 , T2 ), the bounds
in (18)

 and (40) can be used to show
that Q3 = O kY k2 +  . By selecting W 1 large enough,
it can be ensured that kY k2 < kY k1 , and hence, Q3 < Q2 ,
which implies
 vι3 < ι2 . Provided T2 satisfies (37), then
V 2 − vv ι2 e− v (T2 −T1 ) ≤ vv ι2 , which implies V 3 = vv ι2 ,
and hence, V 3 < V 2 and θ3 < θ2 . Therefore, the gain
conditions in (28), (29), and (34) are satisfied over [T2 , T3 ).
Since the gain conditions are satisfied, a Lyapunov-based
q
analysis similar to (32)-(36) yields kY (t)k ≤ 2wλ
ww , ∀t ∈
[T2 , T3 ). Given any ε > 0, the gains α, β, and k
can be selected large enough to satisfy kY k2 ≤ ε,
and hence, kY (t)k ≤ ε, ∀t ∈ [T2 , T3 ) . Furthermore,
a similar Lyapunov-based analysis as (26) - (30) yields
v
V (Z (t) , t) ≤ V 3 − vv ι3 e− v (t−T2 ) + vv ι3 , ∀t ∈ [T2 , T3 ). If
T3 = ∞ then lim supt→∞ V (Z (t) , t) ≤ vv ι3 , which, from

2
0
-2
0

10

20

30

40

50

Time (s)
Fig. 2. Trajectories of the parameter estimation errors using noise-free
position measurements.



2
3k2 2
Q3 = O kY k2 +  and ι3 = k + 2aθ Q3 implies that
lim supt→∞ kZ (t)k = O (ε + ).
If T3 6= ∞ then an inductive continuation of the
Lyapunov-based analysis to the time intervals [Ts−1 , Ts )
shows that provided the dwell time Ts satisfies (37), the gain
conditions in (28), (29), and (34) are satisfied for all t > T3 ,
the state Y satisfies
kY (t)k ≤ ε, ∀t > T1 ,

(41)

and Qs ≤ Qs−1 , ιs ≤ ιs−1 , V s ≤ V s−1 , and θs ≤ θs−1 ,
for all s > 3.
The bound in (41) and the fact that Qs
=

O kY ks−1 +  indicate that Qs = O (ε + ) , ∀s ∈ N.
 v
Furthermore, V (Z (t) , t) ≤ V s − vv ιs e− v (t−Ts−1 ) + vv ιs ,
∀t ∈ [Ts−1 , Ts ), ∀s ∈ N, which, along with the dwell time
requirement, implies that lim supt→∞ V (Z (t) , t) ≤ vv ιs ,
and hence, lim supt→∞ kZ (t)k = O (ε + ).
VII. S IMULATION
The developed technique is simulated using a model for
a two-link robot manipulator arm. The uncertainty g (x, u)
is linearly parameterizable as g T (x, u) = θT σ (x, u) . That

1

1.5

0.5

1

0

0.5

-0.5

0

-1

-0.5

-1.5

-1

-2

-1.5

-2.5

-2
0

10

20

30

40

0

50

10

30

40

50

Time (s)

Time (s)
Fig. 3. Trajectories of the generalized position estimation errors using
noise-free position measurements.

20

Fig. 6. Trajectories of the generalized position estimation errors with a
Gaussian measurement noise (variance = 0.001).

6
6
4
4
2
2
0
0
-2
-2
-4
-4
-6
-6
-8
0

-8
0

10

20

30

40

50

10

20

30

40

50

Time (s)

Time (s)
Fig. 4. Trajectories of the generalized velocity estimation errors using
noise-free position measurements.

Fig. 7. Trajectories of the generalized velocity estimation errors with a
Gaussian measurement noise (variance = 0.001).

5

is, the selected model belongs to a sub-class of systems
defined by (1), where the function approximation error, ε,
is zero. Since the ideal parameters, θ, are uniquely known,
the selected model facilitates quantitative analysis of the
parameter estimation error. The dynamics of the arm are
described by (1), where

0

f 0 (x, u) = − (M (p)) Vm (p, q) q + (M (p)) u,


T
−1
−1
g T (x, u) = θT
D (q) .
(M (p))
(M (p))
(42)

10

−1

-5
0

10

20

30

40

50

Time (s)
Fig. 5. Trajectories of the parameter estimation errors with a Gaussian
measurement noise (variance = 0.001).

−1

In (42), u
∈
R2 is the control input,
D
,
diag [tanh (q1 ) , tanh (q2 )], M (p)
,
 (q)
p1 + 2a3 c2 (p) , a2 + a3 c2 (p)
, and Vm (p, q)
,
a2
 a2 + a3 c2 (p) ,

−a3 s2 (p) q2 , −a3 s2 (p) (q1 + q2 )
, where c2 (p) =
a3 s2 (p) q1 ,
0
cos (p2 ) , s2 (p) = sin (p2 ), and a1 = 3.473, a2 = 0.196,
and a3 = 0.242 are constants. The system has four unknown

parameters.
The ideal values of the unknown parameters are

T
θ = 5.3 1.1 8.45 2.35 .
The contribution of this paper is the design of a parameter
estimator and a velocity observer. The controller is assumed
to be any controller that results in bounded system response.
In this simulation study, the controller, u, is designed so that
the system tracks the trajectory p1 (t) = p2 (t) = sin (3t) +
sin (2t).
The simulation is performed using Euler forward numerical integration using a sample time of Ts = 0.0005
2
values of the generalized position, p,
seconds. Past τ1T+τ
s
and the control input, u, are stored in a buffer. The matrices
P , Ĝ, and F̂ for the parameter update law in (19) are
computed using trapezoidal integration of the data stored
in the aforementioned buffer. Values of P , Ĝ, and F̂ are
stored in the history stack and are updated according to the
algorithm detailed in Fig. 1.
The initial estimates of the unknown parameters are selected to be zero, and the history stack is initialized so that all
the elements of the history stack are zero. Data is added to the
history stack using a singular value maximization algorithm.
To demonstrate the utility of the developed method, three
simulation runs are performed. In the first run, the observer
is assumed to have access to noise free measurements of the
generalized position. In the second run, a zero-mean Gaussian noise with variance 0.001 is added to the generalized
position signal to simulate measurement noise. The values of
various simulation parameters selected for the three runs are
provided in Table I. Figure 2 demonstrates that in absence
of noise, the developed parameter estimator drives the state
estimation error, x̃, and the parameter estimation error, θ̃,
close to the origin. Figures 5 - 7 indicate that the developed
technique can be utilized in the presence of measurement
noise, with expected degradation of performance.
VIII. C ONCLUSION
This paper develops a concurrent learning based adaptive
observer and parameter estimator to simultaneously estimate
the unknown parameters and the generalized velocity of
second-order nonlinear systems using generalized position
measurements. The developed technique utilizes a dynamic
velocity observer to generate state estimates necessary for
data-driven adaptation. A purging algorithm is developed to
improve the quality of the stored data as the state estimates
converge to the true state. By integrating n−times, the
developed method can be generalized to higher-order linear
systems.
Simulation results indicate that the developed method is
robust to measurement noise. A theoretical analysis of the
developed method under measurement noise and process
noise is a subject for future research. Future efforts will
also focus on the examination the effect of the integration
intervals, τ1 and τ2 , on the performance of the developed
estimator.
R EFERENCES
[1] P. Ioannou and J. Sun, Robust Adaptive Control. Prentice Hall, 1996.

[2] S. Sastry and M. Bodson, Adaptive Control: Stability, Convergence,
and Robustness. Upper Saddle River, NJ: Prentice-Hall, 1989.
[3] M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and
Adaptive Control Design. New York, NY, USA: John Wiley & Sons,
1995.
[4] M. A. Duarte and K. Narendra, “Combined direct and indirect approach to adaptive control,” IEEE Trans. Autom. Control, vol. 34,
no. 10, pp. 1071–1075, Oct 1989.
[5] M. Krstić, P. V. Kokotović, and I. Kanellakopoulos, “Transientperformance improvement with a new class of adaptive controllers,”
Syst. Control Lett., vol. 21, no. 6, pp. 451–461, 1993.
[6] G. Chowdhary and E. Johnson, “A singular value maximizing data
recording algorithm for concurrent learning,” in Proc. Am. Control
Conf., 2011, pp. 3547–3552.
[7] G. Chowdhary, T. Yucelen, M. Mühlegg, and E. N. Johnson, “Concurrent learning adaptive control of linear systems with exponentially
convergent bounds,” Int. J. Adapt. Control Signal Process., vol. 27,
no. 4, pp. 280–301, 2013.
[8] S. Kersting and M. Buss, “Concurrent learning adaptive identification
of piecewise affine systems,” in Proc. IEEE Conf. Decis. Control, Dec.
2014, pp. 3930–3935.
[9] G. Chowdhary, M. Mühlegg, J. How, and F. Holzapfel, “Concurrent
learning adaptive model predictive control,” in Advances in Aerospace
Guidance, Navigation and Control, Q. Chu, B. Mulder, D. Choukroun,
E.-J. van Kampen, C. de Visser, and G. Looye, Eds. Springer Berlin
Heidelberg, 2013, pp. 29–47.
[10] H. Modares, F. L. Lewis, and M.-B. Naghibi-Sistani, “Integral reinforcement learning and experience replay for adaptive optimal control
of partially-unknown constrained-input continuous-time systems,” Automatica, vol. 50, no. 1, pp. 193–202, 2014.
[11] R. Kamalapurkar, J. Klotz, and W. E. Dixon, “Concurrent learningbased online approximate feedback Nash equilibrium solution of
N -player nonzero-sum differential games,” IEEE/CAA J. Autom. Sin.,
vol. 1, no. 3, pp. 239–247, Jul. 2014, Special Issue on Extensions of
Reinforcement Learning and Adaptive Control.
[12] B. Luo, H.-N. Wu, T. Huang, and D. Liu, “Data-based approximate
policy iteration for affine nonlinear continuous-time optimal control
design,” Automatica, 2014.
[13] R. Kamalapurkar, P. Walters, and W. E. Dixon, “Modelbased reinforcement learning for approximate optimal regulation,”
Automatica, vol. 64, pp. 94–104, Feb. 2016.
[14] T. Bian and Z.-P. Jiang, “Value iteration and adaptive dynamic
programming for data-driven adaptive optimal control design,” Automatica, vol. 71, pp. 348–360, 2016.
[15] R. Kamalapurkar, J. A. Rosenfeld, and W. E. Dixon, “Efficient
model-based reinforcement learning for approximate online optimal
control,” Automatica, vol. 74, pp. 247–258, Dec. 2016.
[16] R. Kamalapurkar, B. Reish, G. Chowdhary, and W. E. Dixon,
“Concurrent learning for parameter estimation using dynamic
state-derivative estimators,” IEEE Trans. Autom. Control, 2017, to
appear.
[17] A. Parikh, R. Kamalapurkar, and W. E. Dixon, “Integral concurrent
learning: Adaptive control with parameter convergence without PE or
state derivatives,” 2017, submitted, see arXiv:1512.03464, automatica.
[18] R. Kamalapurkar, “Online output-feedback parameter and state estimation for second order linear systems,” in Proc. Am. Control Conf.,
2017, to appear, see also, arXiv:1609.05879.
[19] H. T. Dinh, R. Kamalapurkar, S. Bhasin, and W. E. Dixon,
“Dynamic neural network-based robust observers for uncertain
nonlinear systems,” Neural Netw., vol. 60, pp. 44–52, Dec. 2014.
[20] B. Xian, M. S. de Queiroz, D. M. Dawson, and M. McIntyre, “A
discontinuous output feedback controller and velocity observer for
nonlinear mechanical systems,” Automatica, vol. 40, no. 4, pp. 695–
700, 2004.
[21] G. Chowdhary, “Concurrent learning for convergence in adaptive
control without persistency of excitation,” Ph.D. dissertation, Georgia
Institute of Technology, Dec. 2010.
[22] H. K. Khalil, Nonlinear Systems, 3rd ed. Upper Saddle River, NJ:
Prentice Hall, 2002.

