1

Memory-Based Data-Driven MRAC Architecture
Ensuring Parameter Convergence

arXiv:1602.00482v1 [] 1 Feb 2016

Sayan Basu Roy, Shubhendu Bhasin, Indra Narayan Kar

Abstract—Convergence of controller parameters in standard
model reference adaptive control (MRAC) requires the system
states to be persistently exciting (PE), a restrictive condition to
be verified online. A recent data-driven approach, concurrent
learning, uses information-rich past data concurrently with
the standard parameter update laws to guarantee parameter
convergence without the need of the PE condition. This method
guarantees exponential convergence of both the tracking and
the controller parameter estimation errors to zero, whereas,
the classical MRAC merely ensures asymptotic convergence of
tracking error to zero. However, the method requires knowledge
of the state derivative, at least at the time instances when the
state values are stored in memory. The method further assumes
knowledge of the control allocation matrix. This paper addresses
these limitations by using a memory-based finite-time system
identifier in conjunction with a data-driven approach, leading to
convergence of both the tracking and the controller parameter
estimation errors without the PE condition and knowledge
of the system matrices and the state derivative. A Lyapunov
based stability proof is included to justify the validity of the
proposed data-driven approach. Simulation results demonstrate
the efficacy of the suggested method.
Index Terms—MRAC, CL, Data-Driven, Parameter Convergence, PE.

I. I NTRODUCTION
HE design objective of Model Reference Adaptive Control (MRAC) is to make the system imitate the response
of a chosen reference model. Classical and many recent
MRAC techniques that use merely instantaneous data for
adaptation (see [1]–[5] and references there in) require that
the system states be persistently exciting (PE) to ensure the
convergence of the parameter estimates to their true values
[3]. In [6], Boyd and Sastry proved that the PE condition on
the regressor translates to the reference input having as many
spectral lines as the number of unknown parameters, however,
the condition is rather restrictive. Enforcing the PE condition
through exogenous excitation of the input is not always
realizable and it is often impractical to monitor online whether
a signal will remain PE as the condition depends on the future
values of the signal. Since parameter convergence under the
PE condition is difficult to apply, various algorithms like emodification, σ-modification etc. are proposed in literature to
guarantee boundedness of the parameters [2], [7].
Similar to MRAC, other model-based control methods such
as model-based reinforcement learning (MBRL) [8]–[12], and
model-based predictive control (MPC) [13]–[16], require the

T

Sayan Basu Roy, Shubhendu Bhasin and Indra Narayan Kar are with
the Department of Electrical Engineering, Indian Institute of Technology
Delhi, New Delhi, India e-mail:(sayanetce@gmail.com, sbhasin@ee.iitd.ac.in,
ink@ee.iitd.ac.in).

controller to be developed based on the estimates of the
unknown parameters. Therefore, the stability of the closedloop system and the performance of the control law in all
these cases crucially rely on parameter convergence, which
requires restrictive PE condition.
Recent works [17]- [18] on learning and data-driven control
methods have shown promise in improving tracking performance as they use input-output data along the system trajectory which carries sufficient information about the unknown
system and the controller parameters. Girish et al. [18]–[21]
proposed a novel approach, coined as concurrent learning
(CL), where information-rich past data is stored and concurrently used along with gradient based parameter update laws.
Although the parameter estimation error is not directly measurable, the intelligent introduction of the concurrent learning
term computed from the past stored data is proportional to
parameter estimation error. A sufficient condition associated
with the rank of a matrix formed out of stored data is
required for parameter convergence. Unlike the PE condition,
the rank condition is more realistic and guarantees exponential
convergence of tracking and parameter estimation errors to
zero. Moreover, CL-based techniques are employed in the
context of adaptive optimal control in [11], [22], [23] and
experimental success have been found in [24], [25].
Although concurrent learning is a powerful online adaptive
control method, it requires the state derivative information at
the time points at which the state values are stored [19]. In
many practical situations, the state derivative is not measurable. In [19], an optimal fixed point smoothing technique is
used to estimate derivative at past values using a forward
and backward Kalman filter [26]. However, the estimation
method requires storing several forward and backward data
points in time leading to a high memory requirement. Further,
the state derivative estimation error degrades the exponential
convergence result to a weaker one of uniformly ultimately
bounded (UUB) stability. Moreover, both the classical MRAC
and the concurrent learning laws require knowledge of the B
matrix (in the standard state space realization).
In [27]–[30] the uncertainty in the control allocation matrix
(B in LTI framework) is dealt with in different ways, however,
few results in literature tackle the general case of controlling
dynamical systems when the knowledge of input matrix is
absent. Some recent results [31], [32] have attempted to
address the limitations in the concurrent learning framework.
In [31], the authors designed a control law without requiring
B, however, the knowledge of the A matrix and the state
derivative is required. The requirement of the state derivative
is avoided in [32] by the introduction of a dynamic state

2

derivative estimator, which leads to a UUB result, while
requiring knowledge of the B matrix.
The contribution of this paper is to achieve the MRAC goal
with parameter convergence, using only state and input data.
In this work, the system matrices A and B are considered
to be unknown and the state derivative information is also
not available. Two memory stacks to store effective past data
points are utilized to solve the data-driven MRAC problem,
relaxing the assumption of knowledge of the state derivative
and the input B matrix. Using sufficient rank conditions
on the matrices formed out of the stored data of memory
stacks, finite-time identification of the system parameters and
subsequent exponential convergence of tracking and controller
parameter estimation errors is obtained. The identification
method proposed in this work is inspired from [33], [34]. The
finite time identification of system parameters eliminates the
need of the computationally burdensome purging algorithm
[31]. By introducing an additional gain parameter in the
parameter update law, this work further avoids the singular
value maximisation algorithm [35], used to continuously update the history stack for accelerated convergence. Moreover,
it is proved that the aforementioned rank condition on the
respective matrices merely demands the corresponding signals
to be exciting for a finite time interval, which is less restrictive
than the PE condition. Unlike PE, the rank condition is
required for the matrices formed out of past stored signals
and therefore can be verified online.

Consider a continuous-time LTI system given by
(1)

where x(t) ∈ Rn denotes the state and u(t) ∈ Rd denotes
the control input to the system and A ∈ Rn×n , B ∈ Rn×d
are the system matrices. It is assumed that the pair (A, B) is
controllable and that B has full column rank. 1
A reference model is chosen as follows to characterise the
desired closed loop response of the system (1).
ẋm (t) = Am xm (t) + Bm r(t)

(2)

where Am ∈ Rn×n is Hurwitz, xm (t) ∈ Rn is the model
state and r(t) ∈ Rd denotes a bounded, piecewise continuous
reference input signal. An adaptive control law, comprising a
linear feedback term and a linear feedforward term, is defined
as [2]
u(t) = KxT (t)x(t) + KrT (t)r(t)
(3)
where Kx (t) ∈ Rn×d and Kr (t) ∈ Rd×d . Substituting (3) in
(1) yields
ẋ = (A + BKxT )x(t) + BKrT r(t)

A + BKx∗T = Am
BKr∗T

= Bm

(4)

To facilitate the design objective of making system (4) respond
as the chosen reference model of (2), the following matching
condition is introduced [2], [3].
1 Typically in physical systems d ≤ n and the above mentioned condition
is satisfied.

(5)
(6)

Using (5) and (6), the closed-loop system in (4) can be
written as
ẋ = Am x + Bm r + B K̃xT x + B K̃rT r

(7)

where K̃x , Kx − Kx∗ and K̃r , Kr − Kr∗ . The tracking
error is defined as
e(t) , x(t) − xm (t)

(8)

Using (2), (7) and (8), the error dynamics is obtained as
ė = Am e + B K̃xT x + B K̃rT r

(9)

The standard adaptive update laws for Kx (t) and Kr (t) are
given as [2]
K̇x = −Γx xeT P B
T

K̇r = −Γr re P B

(10)
(11)

where Γx > 0 and Γr > 0 denote positive definite learning
rate matrices of appropriate dimension and P ∈ Rn×n is a
positive definite matrix satisfying the Lyapunov equation
ATm P + P Am + Q = 0

II. C LASSICAL MRAC

ẋ(t) = Ax(t) + Bu(t)

Assumption 1. There exists Kx∗ ∈ Rn×d and Kr∗ ∈ Rd×d
such that

(12)

for any given positive definite Q ∈ Rn×n .
The equations (3) and (10)-(11) are the classical MRAC
laws, which guarantee the tracking error e(t) → 0 as t → ∞.
However, convergence of the controller parameters Kx (t) and
Kr (t) to their true values (Kx∗ and Kr∗ respectively) is only
guaranteed if a restrictive PE condition is satisfied [2], [4],
[36]. The persistence of excitation of a vector signal x(t) is
defined below [3].
Definition 1. A bounded vector signal x(t) is exciting over
an interval [t, t + T ] , T > 0 and t ≥ t0 if ∃ α > 0 such that
the following condition holds:
Z t+T
x(τ )xT (τ )dτ ≥ αI
t

where I denotes an identity matrix.
Definition 2. A bounded vector signal x(t) is persistently
exciting (PE) if ∀t ≥ t0 , ∃ T > 0 and α > 0 such that:
Z t+T
x(τ )xT (τ )dτ ≥ αI
t

According to [6], for the system states of (1) to be PE, the
reference signal r(t) must be sufficiently rich i.e. it must contain n distinct frequencies if there are 2n unknown parameters.
The PE condition is restrictive and difficult to verify online as
it relies on the future behaviour of the dynamical systems.

3

III. M EMORY-BASED DATA -D RIVEN MRAC
This section proposes a memory-based data-driven architecture for model reference adaptive control and identification of
unknown LTI systems with controller parameter convergence.
This architecture builds on the concurrent learning technique
[18], [19], [37], [38], [39] which utilizes past recorded data
concurrently with the current data for adaptation. The concurrent learning method fruitfully utilises a memory stack for
storing the state x(t) and the reference signal r(t) at different
time points. A full rank condition on the matrices formed
out of the memory stack guarantees exponential convergence
of both the tracking and the controller parameter estimation
errors. However, both classical MRAC and the concurrent
learning frameworks assume knowledge of the input B matrix.
Concurrent learning, additionally, requires the state derivative
information at least at the time instances when the state
information is stacked in memory. Although there have been
efforts to overcome these pitfalls, the solutions given by [31],
[32] are partial.
Motivated by the aforementioned limitations, this work
achieves the MRAC goal only from input and state data. The
system matrices (A and B) as well as the state derivative information is considered to be unknown in this framework. The
intelligent exploitation of multiple memory stacks results in
finite-time identification of system parameters and subsequent
exponential convergence of tracking and controller parameter
estimation errors to zero.
A. Finite Time Identification of System Parameters
The plant dynamics in (1) can be linearly parametrized as
ẋ = Y (x, u)θ

(13)

where Y (x, u) ∈ Rn×n(n+d) is the regressor matrix and θ ∈
Rn(n+d) is a column vector of all the elements of A and B
defined as


vec(AT )
θ,
vec(B T )

where vec(Z) ∈ Rab denotes the vectorization of a matrix Z ∈
Ra×b , obtained by stacking the columns of the matrix Z. The
set of equations required for system parameter identification
are described as [33]
˙
(14)
x̂˙ = Y θ̂ + km x̃ + mθ̂
˙
θ̂ = kθ mT (x̃ − γ)
(15)
ṁ = Y − km m,

m(t0 ) = 0

(16)

where m(t) ∈ Rn×n(n+d) and km , kθ are positive scalar gains
and x̃(t) and γ(t) ∈ Rn are defined as
x̃ , x − x̂

(17)

γ , x̃ − mθ̃

(18)

with γ(t0 ) = x̃(t0 ).
Although γ(t) in (18) is unmeasurable, the use of the initial
condition m(t0 ) = 0 in (16) ensures that γ(t) is available
online by solving (20) with known initial condition.
An auxiliary variable g(t) ∈ Rn is defined as
g , mθ̂ + x̃ − γ

(21)

Substituting γ from (18) in (21) leads to
g(t) = m(t)θ,

∀t ≥ t0

(22)

The strategic introduction of variables m(t) and γ(t) transforms the system in (13) to a standard linear regression form
of (22), where both m(t) and g(t) are known signals, obviating
the need of ẋ(t) information.
Consider a memory stack W = {(mj , gj )}pj=1 updated
online with the signals m(t) and g(t) respectively, where each
mi and gi are collected and stored in W at t = twi with
t0 ≥ tw1 > tw2 > ....twp = tc .2 Here, p is the memory stack
length, which satisfies the condition p ≥ n + d, where n is
the dimension of the state x(t) and d is the dimension of the
input u(t).
Assumption 2. The matrix M , [mT1 , mT2 , ...mTp ]T
Rnp×n(n+d) is full rank i.e. rank(M ) = n(n + d).

∈

This rank condition is analogous to those of CL-based
frameworks [19], [18] of adaptive control. The difference here
is that it is stated in terms of a newly introduced variable m(t)
required for system identification purpose. Unlike the dependency on future behaviour of signals in PE condition, this rank
condition relies on past data. Therefore, this condition can be
easily verified online making it more practical as compared
to the PE condition. The following Lemma establishes only
exciting condition (not PE) of relevant signals as a necessary
condition for the Assumption 2 to hold.
Lemma 1. The necessary condition for the matrix M to be
full rank is the state x(t) and the input u(t) be exciting over
the interval t ∈ [t0 , tc ] as per Definition 1.3
Theorem 1. The system parameter identification error kθ̃(t)k
is non-increasing during the interval [t0 , tc ] using the update
laws (14)-(16). Provided Assumption 2 holds, the stored data
in W can be used to achieve the finite time identification of
system parameters (θ) at t = tc .
Proof: Consider a Lyapunov candidate as
1 T
θ̃ θ̃
2
Differentiating (23) along (15) and using (18) yields
Vθ =

V̇θ = −kθ θ̃T mT mθ̃ ≤ 0

(23)

(24)

(19)

which implies kθ̃(t)k ≤ kθ̃(t0 )k, ∀t ≥ t0 .
For finite time identification of system parameters, define a
matrix G , [g1T , g2T , ...gpT ]T ∈ Rnp×1 .

Differentiating (18) and using (14)-(17), the following expression is obtained
γ̇ = −km γ
(20)

2 The data storing mechanism in this work is analogous to that of [32], [35]
and [38], hence, the details of this mechanism are omitted here.
3 For proof see Appendix

where θ̃(t) is given by
θ̃ , θ − θ̂

4

Since (22) is valid for ∀t ≥ t0 , the following equation is
satisfied.
Mθ = G
(25)

Lemma 3. The necessary condition for the matrix Z to be full
rank is the state x(t) and the reference input r(t) be exciting
over the interval t ∈ [t0 , ts ] as per Definition 1.

The Assumption 2 implies M T M is an invertible matrix and
hence the system parameter θ can be found from the following
least-square like expression.

The controller parameter φ(t) is updated as

T T

for t0 ≤ t ≤ tc

proj(−Γφ z B̂ P e)
T T
−Γ
z
B̂
P
e
for
tc < t <tm
φ̇ =
φ

Pq

T
T
−Γφ z B̂ P e + kφ j=1 zjT ǫKj
for t ≥ tm

θ̂F T (t) = (M T M )−1 M T G = θ,

t ≥ tc

(26)

Using (26),at t = tc , finite time convergence of θ is obtained.
The identification equations (14)-(16) and (21) are merely
required for t ∈ [t0 , tc ]. At t = tc , using (26) finite time
identification of system parameters is achieved using the
memory stack W . The finite-time (FT) identifier is given by
(
θ̂(t) if t < tc
θ̂F T (t) =
(27)
θ
if t ≥ tc
The FT identifier method requires instantaneous inversion of
M T M ∈ Rn(n+d)×n(n+d) at t = tc , which may be impractical
for large dimensional systems. However, the following Lemma
shows that the matrix M T M is significantly sparse and
sparsity increases with the state dimension n implying that
sophisticated techniques can be applied for fast computation
of the inverse.
T

Lemma 2. The fraction of non-zero elements in M M is
where n is the state dimension.

1
n,

B. Tracking Error and Controller Parameter Convergence
The control law in (3) can be linearly parametrized as
u = z(x, r)φ(t)

(28)

d×d(n+d)

d(n+d)

where z(x, r) ∈ R
and φ ∈ R
is a column
vector consisting of all the elements of Kx (t) and Kr (t)
defined as


vec(Kx )
φ,
vec(Kr )
Using (28), the error dynamics in (9) can be written as
ė = Am e + Bz φ̃

(29)

where φ̃ = φ − φ∗ and
φ ,
∗



vec(Kx∗ )
vec(Kr∗ )



For the convergence of tracking and controller parameter
estimation errors, another memory stack H = {xj , rj }qj=1 is
populated online by state and input signals, respectively where
each xj and rj is stored at t = thj with t0 ≥ th1 > th2 >
.. > tzq = ts . Each pair (xj , rj ) is referred to as a data point
and q is the length of the stack satisfying q ≥ n ≥ d.

(30)

where Γφ ∈ Rd(n+d)×d(n+d) is a positive definite learning
rate matrix, proj denotes projection operator [40] which
ensures parameter boundedness within a convex region in the
parameter space and tm = max(tc , ts ). Further, kφ is a scalar
gain introduced to alter the rate of convergence and B̂(t) is
extracted from θ̂F T (t) of (27). The error variable ǫKj (t) in
(30) is defined as
ǫKj (t) = ǫKxj (t) + ǫKrj (t)

(31)

with the following two expressions.
ˆj − Am xj − Bm rj − B̂ǫKr (t))
ǫKxj (t) , (B̂ T B̂)−1 B̂ T (ẋ
j
(32)
ǫKrj (t) , KrT (t)rj − (B̂ T B̂)−1 B̂ T Bm rj

(33)

ˆj = Yj θ̂F T .
where ẋ
Theorem 2. For the system (1), the control law in (28) and
the update laws in (30) along with the finite-time system
identifier θ̂F T (t) (27) ensure boundedness of the tracking and
the controller parameter estimation errors for t ∈ [t0 , tm ) and
the global exponential convergence of those errors to zero is
guaranteed for t ≥ tm , provided the Assumption 3 is satisfied.
Proof: Consider the following Lyapunov candidate
Vξ =

1 T
ξ Λξ
2

(34)

where ξ(t) , [eT (t), φT (t)]T and


P
0n×d(n+d)
Λ,
0d(n+d)×n
Γ−1
φ
Taking time derivative of (34) along the trajectories of (29) and
(30) the following expression is obtained during t ∈ [t0 , tc ]
1
V̇ξ ≤ − eT Qe + eT P BZ φ̃ − φ̃T z T B̂ T P e
2

(35)

Assumption 3. The matrix Z = [z1T , z2T , ..., zqT ]T is full rank
i.e. rank(Z) = d(n + d), where zj = z(xj , rj ) for j = 1(1)q.

The inequality occurs due to the use of projection operator
(For details see [41]). The above inequality can be further
modified as
1
(36)
V̇ξ ≤ − eT Qe + eT P B̃Z φ̃
2

This assumption is analogous to Assumption 2. Here it
is stated in terms of z(t), which is relevant to controller
parameter convergence. The following Lemma provides a
necessary condition for the Assumption 3 to hold.

where B̃ , B − B̂. Using the fact Z φ̃ = K̃x x + K̃r r,
yields
1
V̇ξ ≤ − β1 kek2 + β2 kek
(37)
2

T

T

5

where
β1 = λmin (Q) − 2kP kkB̃kkK̃x k

(38)

β2 = kP kkB̃k(kK̃x kkxm k + kK̃r kkrk)

(39)

λmin (.), in (38), denotes the minimum eigen value of the
corresponding argument matrix. In (39), kK̃x (t)k and kK̃r (t)k
are bounded by the use of projection operator [40] in (30) and
kB̃(t)k ∈ L∞ from Theorem 1. As r(t) ∈ L∞ and Am is
Hurwitz by definition, xm (t) ∈ L∞ , implying β1 (t) ∈ L∞
and β2 (t) ∈ L∞ with β2 > 0. However, the sign of β1 is
uncertain during t ∈ [t0 , tc ]. It can be inferred that once β1 (t)
becomes greater than zero, it will remain greater than zero as
kθ̃(t)k is non-increasing in the interval [t0 , tc ] as per Theorem
1. Moreover, β1 = λmin (Q) > 0 at t = tc as kθ̃(t)k = 0
at t = tc . Three cases are possible depending on the θ̃(t)
dynamics.
case 1: β1 (t) ≤ 0, ∀t ∈ [t0 , tc )
case 2: β1 (t) > 0, ∀t ∈ [t0 , tc ]
case 3: β1 (t) ≤ 0, ∀t ∈ [t0 , tf ] and β1 (t) > 0, ∀t ∈ (tf , tc ]
The three cases are analysed separately as follows.
case 1: As β1 (t) ≤ 0, it is hard to comment on the
exact bound of the tracking error. However, finite tracking
error can be claimed from (4), which can be expressed as
ẋ = Ā(t)x(t) + ḡ(t), where Ā(t) = A + BKxT (t) and
ḡ(t) = BKrT (t))r(t). As equation (4) is a linear equation
in x(t) with kĀ(t)k and kḡ(t)k are bounded i.e. Ā(t) ∈ L∞
and ḡ(t) ∈ L∞ in the finite time interval [t0 , tc ], using Global
Existence and Uniqueness theorem [42] it can be argued that
x(t) cannot have a finite escape time. Therefore x(t) ∈ L∞
in finite time (t ≤ tc ) if x(t0 ) is finite, leading to the tracking
error e(t) ∈ L∞ as xm (t) ∈ L∞ , implying ξ(t) ∈ L∞ .
case 2: As β1 (t) > 0, (37) can be further modified to
2
β2,max
1
V̇ξ ≤ − β11 kek2 +
2
4β12

(40)

where β11 + β12 = β1,min > 0 with β11 > 0, β12 > 0 and
β2,max can be found by upper bounding every time-varying
term in (39). Due to the use of proj in (30) during t ∈ [t0 , tc ],
Vξ in (34) can be upper bounded as
Vξ ≤

1
λmax (P )kek2 + D
2

β11
β11 D
2,max
where η1 = λmax
(P ) and η2 =
4β12 + λmax (P ) . Using
comparison Lemma [42], the above differential inequality
results in the following UUB condition.


η2
η2
exp (−η1 (t − t0 )) + , ∀t ∈ [t0 , tc ]
Vξ (t) ≤ Vξ (t0 ) −
η1
η1
(43)
The inequality in (43) implies ξ(t) ∈ L∞ during the same
interval via Theorem 4.18 of [42].
The analysis done in case 1 holds independent of the sign of
β1 (t). However, in case of β1 (t) > 0, it is possible to get an
4 kφk2
max

is defined by the convex region in the projection operator.

1
V̇ξ = − eT Qe ≤ 0
2

(45)

Thus Vξ (t) is non-increasing in this interval, implying
Vξ (tm ) ≤ Vξ (tc ). Again, using Theorem 4.18 of [42] it can be
inferred that ξ(t) ∈ L∞ during t ∈ (tc , tm ) as Vξ (t) ∈ L∞ .
For t ≥ tm , using θ̂F T = θ from Theorem 1, the time
derivative of (34) along (29) and (30) can be expressed as
1
V̇ξ = − eT Qe + eT P Bz φ̃
2


q
X
zjT zj  φ̃
− φ̃T z T B T P e − φ̃T kφ

(46)

j=1

Using (6) and (33), ǫTKr (t) can be expressed as
j

ǫKrj (t) = K̃rT (t)rj

(47)

and using (7), (32) and (49) it can be shown that
ǫKxj (t) = K̃xT (t)xj

(48)

Further (31), (47) and (48) lead to
ǫKj (t) = zj φ̃(t)

(49)

The expression (49) is used to derive (46), which can be further
upper bounded as
1
V̇ξ ≤ − λmin (Q)kek2 − kφ λmin (Ωz )kφ̃k2
2

(41)

where D = 12 λmin (Γφ )kφk2max .4 Further upper bounding (40)
using (41) yields
V̇ξ ≤ −η1 Vξ + η2
(42)
β2

exact expression of bound for the Lyapunov function Vξ (t) as
shown in (43).
case 3: During t ∈ [t0 , tf + ǫ), following the arguments
similar to case 1, it can be established that ξ(t) ∈ L∞ , where
ǫ > 0 is infinitesimally small. Further, during t ∈ [tf + ǫ, tc ],
following the analysis similar to case 2, the following bound
can be derived.


η2
η2
exp (−η1 (t − tβ )) + , ∀t ∈ [tβ , tc ]
Vξ (t) ≤ Vξ (tβ ) −
η1
η1
(44)
with tβ = tf + ǫ, implying ξ(t) ∈ L∞ in the same interval.
In the interval t ∈ (tc , tm ) using B̃ = 0 from Theorem 1 in
the time derivative of (34) along (29) and (30)

where
Ωz =

q
X

zjT zj = Z T Z

(50)

(51)

j=1

Hence, based on Assumption 3, Ωz > 0 which implies V̇ξ ≤ 0.
Further from (50) the following bound can be obtained
V̇ξ ≤ −βVξ

(52)

where β is given by
β=

min(λmin (Q), 2kφ λmin (Ωz ))
max(λmax (P ), λmax (Γ−1
φ ))

The differential inequality in (52) leads to the subsequent
exponentially convergent bound
Vξ (t) ≤ Vξ (tm )e−β(t−tm ) , ∀t ∈ [tm , ∞)

(53)

6

Remark 2. To improve the rate of convergence, the memory
stack is continuously updated in [35] using an algorithm to
maximize the minimum singular value of a matrix analogous
to Z. To avoid computational burden associated with the
continuous stack update, the proposed algorithm updates the
stack until the sufficient rank condition is satisfied. The speed
of convergence is controlled by appropriately choosing Q, kφ
and Γφ . Moreover, since θ is obtained in finite time using (14)(22) and the memory stack W , the need for computationally
involved purging algorithm [31] is obviated.
Remark 3. The proposed memory-based data-driven technique for parameter convergence is similar to classical integral control in the following sense. It is well-known that
integral control, which captures the effect of entire past of
the relevant signal, reduces steady state error. The proposed
approach also stores information-rich past data (although not
the entire past) in the memory stack using a non-linear sampling technique and utilizes the stacked data in the parameter
update law, leading to exponential convergence of parameter
estimation error to zero. Future research can be carried out
to investigate the relation between the proposed method and
the classical integral control.
IV. S IMULATION R ESULTS
To demonstrate the effectiveness of the proposed data driven
technique, a second order linear plant is considered.


 
0 1
0
A=
B=
5 2
2
The reference model matrices are considered as


 
0
1
0
Am =
Bm =
−8 −10
1
Note that Am is a Hurwitz matrix but A is not. Using the
matching conditions (5) and (6) Kx∗ is equal to [−6.5, −6]T
and Kr∗ is 0.5. The reference signal r(t) is chosen as
r(t) = 20e−t/2
which is a non-PE signal. The matrix Q of Lyapunov equation
(12) is selected as


5 0
Q=
0 5
The adaptation gains Γx and Γr are chosen as Γx = I2 and
Γr = I1 . The gain parameters are chosen as kθ = 80, km = 10
and kφ = 40.
The plot of the error dynamics of system parameters is
shown in Fig 1. The tracking error plot is shown in Fig 2,

error
error
error
error
error
error

System parameter estimation error

3

2

in
in
in
in
in
in

A(1,1)
A(1,2)
A(2,1)
A(2,2)
B(1)
B(2)

1

0

−1

−2

0

0.01

0.02
0.03
time (sec.)

0.04

0.05

System parameter estimation error θ̃(t)

Fig. 1.

0.6
e1
e2

0.4

0.2

tracking error e(t)

Remark 1. As described by (30), the update law of φ(t)
follows time-dependent switching with at most 2 (finite) switching instances (tc and tm ). Therefore, the boundedness of ξ(t)
during t ∈ [t0 , tm ) and exponential convergence for t ≥ tm
suffices the analysis.

4

0

−0.2

−0.4

−0.6

−0.8

0

1

2

3

4

5

time (sec.)

Fig. 2.

tracking error e(t)

7
error in Kx1

6

controller parameter estimation error

implying ξ(t) → 0 exponentially fast as t → ∞. Further,
the Lyapunov function in (34) is radially unbounded and no
restriction is imposed on Vξ (tm ), implying global exponential
stability (GES).

error in Kx2
error in Kr

5
4
3
2
1
0
−1
−2
−3

0

0.2

0.4

0.6

0.8

1

time (sec.)

Fig. 3.

Controller parameter estimation error φ̃(t)

depicting the convergence of errors to zero within approximately 4.1 seconds. Fig 3. shows the evolution of estimation
error in controller parameters. At t = tm the transition of
parameter update law from one rule to another leads to nondifferentiability at that time point. After t = tm , the error
dynamics converge to zero exponentially.
V. C ONCLUSION
A memory-based data-driven approach is proposed to solve
the MRAC problem for unknown LTI systems, using only
input and state data. Past data along the system trajectory is
stored and used strategically to guarantee finite-time identification of system parameters (θ), convergence of controller

7

parameters (φ) and tracking error. Unlike the restrictive PE
condition in classical adaptive control, only a rank condition
on the recorded matrices is required to ensure parameter
convergence. Further, the work relaxes two vital assumptions
of CL-based frameworks of adaptive control-knowledge of the
state derivative and the input B matrix, without altering the
exponential convergence result.
A PPENDIX
Lemma 4. A memory stack X is populated with the values
of a continuous signal x(t) ∈ Rn×m at different time points
as described below
X = [x1 , x2 , ....xp ], where mp ≥ n and xi = x(ti ) ∈ Rn×m ,
i = 1(1)p with t1 > t2 > ...tp = te .
If the matrix X is full rank, i.e. rank(X) = n, the signal x(t)
is exciting over the interval [t0 , te ] as per Definition 1.
Rt
Proof: Define C , 0 e x(τ )xT (τ )dτ . Taking limit as
ǫ → 0+ and using the
corollary of fundamental theorem of
R a+ǫ
1
calculus limǫ→0+ 2ǫ
a−ǫ f (y)dy = f (a) for any continuous
function f , the following expression can be obtained
Z
1
1 t1 −ǫ
lim
C = lim
x(τ )xT (τ )dτ
ǫ→0+ 2ǫ
ǫ→0+ 2ǫ 0
p−1
X 1 Z tj+1 −ǫ
x(τ )xT (τ )dτ + XX T
(54)
+
2ǫ
tj +ǫ
j=1
where
Z
1 t1 −ǫ
x(τ )xT (τ )dτ
ǫ→0 2ǫ 0
Z
p−1
X
1 tj+1 −ǫ
+
x(τ )xT (τ )dτ ≥ 0
2ǫ
t
+ǫ
j
j=1
lim+

(55)

and XX T ≥ σmin (X)I where σmin (X) > 0 as X is full
1
C > 0, implying
rank. Thus, it can be inferred that limǫ→0+ 2ǫ
C > 0 as ǫ > 0. Therefore
Z te
x(τ )xT (τ )dτ ≥ αI
(56)
0

with α = λmin (C) > 0.
R EFERENCES
[1] K. S. Narendra and A. M. Annaswamy, “A new adaptive law for
robust adaptation without persistent excitation,” Automatic Control,
IEEE Transactions on, vol. 32, no. 2, pp. 134–145, 1987.
[2] K. S. Narendra and A. M. Annaswamy, Stable adaptive systems. Courier
Corporation, 2012.
[3] G. Tao, Adaptive control design and analysis, vol. 37. John Wiley &
Sons, 2003.
[4] K. J. Åström and B. Wittenmark, Adaptive control. Courier Corporation,
2013.
[5] C. Cao and N. Hovakimyan, “Design and analysis of a novel adaptive
control architecture with guaranteed transient performance,” Automatic
Control, IEEE Transactions on, vol. 53, no. 2, pp. 586–591, 2008.
[6] S. Boyd and S. S. Sastry, “Necessary and sufficient conditions for
parameter convergence in adaptive control,” Automatica, vol. 22, no. 6,
pp. 629–639, 1986.
[7] P. A. Ioannou and P. V. Kokotovic, “Instability analysis and improvement
of robustness of adaptive control,” Automatica, vol. 20, no. 5, pp. 583–
594, 1984.

[8] P. Abbeel, M. Quigley, and A. Y. Ng, “Using inaccurate models
in reinforcement learning,” in International Conference on Machine
Learning, pp. 1–8, ACM, 2006.
[9] D. Mitrovic, S. Klanke, and S. Vijayakumar, “Adaptive optimal feedback
control with learned internal dynamics models,” in From Motor Learning
to Interaction Learning in Robots, pp. 65–84, Springer, 2010.
[10] M. Deisenroth and C. E. Rasmussen, “Pilco: A model-based and dataefficient approach to policy search,” in International Conference on
Machine Learning, pp. 465–472, 2011.
[11] R. Kamalapurkar, P. Walters, and W. Dixon, “Concurrent learningbased approximate optimal regulation,” in Conference on Decision and
Control, pp. 6256–6261, IEEE, 2013.
[12] S. Bhasin, R. Kamalapurkar, M. Johnson, K. G. Vamvoudakis, F. L.
Lewis, and W. E. Dixon, “A novel actor–critic–identifier architecture
for approximate optimal control of uncertain nonlinear systems,” Automatica, vol. 49, no. 1, pp. 82–92, 2013.
[13] H. Fukushima, T.-H. Kim, and T. Sugie, “Adaptive model predictive
control for a class of constrained linear systems based on the comparison
model,” Automatica, vol. 43, no. 2, pp. 301–308, 2007.
[14] V. Adetola, D. DeHaan, and M. Guay, “Adaptive model predictive
control for constrained nonlinear systems,” Systems & Control Letters,
vol. 58, no. 5, pp. 320–326, 2009.
[15] G. Chowdhary, M. Mühlegg, J. P. How, and F. Holzapfel, “Concurrent
learning adaptive model predictive control,” in Advances in Aerospace
Guidance, Navigation and Control, pp. 29–47, Springer, 2013.
[16] A. Aswani, H. Gonzalez, S. S. Sastry, and C. Tomlin, “Provably safe
and robust learning-based model predictive control,” Automatica, vol. 49,
no. 5, pp. 1216–1226, 2013.
[17] F. L. Lewis and D. Vrabie, “Reinforcement learning and adaptive
dynamic programming for feedback control,” Circuits and Systems
Magazine, IEEE, vol. 9, no. 3, pp. 32–50, 2009.
[18] G. Chowdhary and E. Johnson, “Concurrent learning for convergence
in adaptive control without persistency of excitation,” in Conference on
Decision and Control, pp. 3674–3679, IEEE, 2010.
[19] G. Chowdhary, T. Yucelen, M. Mühlegg, and E. N. Johnson, “Concurrent
learning adaptive control of linear systems with exponentially convergent
bounds,” International Journal of Adaptive Control and Signal Processing, vol. 27, no. 4, pp. 280–301, 2013.
[20] G. Chowdhary, M. Mühlegg, and E. Johnson, “Exponential parameter
and tracking error convergence guarantees for adaptive controllers without persistency of excitation,” International Journal of Control, vol. 87,
no. 8, pp. 1583–1603, 2014.
[21] H. A. Kingravi, G. Chowdhary, P. A. Vela, and E. N. Johnson, “Reproducing kernel hilbert space approach for the online update of radial bases
in neuro-adaptive control,” Neural Networks and Learning Systems,
IEEE Transactions on, vol. 23, no. 7, pp. 1130–1141, 2012.
[22] R. Kamalapurkar, P. Walters, and W. E. Dixon, “Model-based reinforcement learning for approximate optimal regulation,” Automatica, vol. 64,
pp. 94–104, 2016.
[23] K. G. Vamvoudakis, M. F. Miranda, and J. P. Hespanha, “Asymptotically
stable adaptive-optimal control algorithm with saturating actuators and
relaxed persistence of excitation,” Neural Networks and Learning Systems, IEEE Transactions on, doi: 10.1109/TNNLS.2015.2487972, 2015.
[24] G. Chowdhary, T. Wu, M. Cutler, N. K. Ure, and J. How, “Experimental
results of concurrent learning adaptive controllers,” in AIAA Guidance,
Navigation, and Control Conference,(Minneapolis, MN), AIAA, 2012.
[25] W. E. Dixon, “Experiments using approximate optimal path following
with concurrent learning,” in American Control Conference, pp. 5083–
5083, IEEE, 2015.
[26] A. Gelb, Applied optimal estimation. MIT press, 1974.
[27] E. Lavretsky, “Combined/composite model reference adaptive control,”
Automatic Control, IEEE Transactions on, vol. 54, no. 11, p. 2692, 2009.
[28] G. Tao, S. M. Joshi, and X. Ma, “Adaptive state feedback and tracking
control of systems with actuator failures,” Automatic Control, IEEE
Transactions on, vol. 46, no. 1, pp. 78–95, 2001.
[29] Z. Han and K. S. Narendra, “New concepts in adaptive control using
multiple models,” Automatic Control, IEEE Transactions on, vol. 57,
no. 1, pp. 78–89, 2012.
[30] A. Somanath, Adaptive control of hypersonic vehicles in presence of actuation uncertainties. PhD thesis, Massachusetts Institute of Technology,
2010.
[31] B. Reish and G. Chowdhary, “Concurrent learning adaptive control for
systems with unknown sign of control effectiveness,” in Conference on
Decision and Control, pp. 4131–4136, IEEE, 2014.
[32] A. Kumar and S. Bhasin, “Data driven mrac with parameter convergence,” in Multi-Conference on Systems and Control, pp. 1662–1667,
IEEE, 2015.

8

[33] V. Adetola and M. Guay, “Finite-time parameter estimation in adaptive
control of nonlinear systems,” Automatic Control, IEEE Transactions
on, vol. 53, no. 3, pp. 807–811, 2008.
[34] V. Adetola and M. Guay, “Performance improvement in adaptive control
of linearly parameterized nonlinear systems,” Automatic Control, IEEE
Transactions on, vol. 55, no. 9, pp. 2182–2186, 2010.
[35] G. Chowdhary and E. Johnson, “A singular value maximizing data
recording algorithm for concurrent learning,” in American Control
Conference, pp. 3547–3552, IEEE, 2011.
[36] P. A. Ioannou and J. Sun, Robust adaptive control. Courier Corporation,
2012.
[37] G. V. Chowdhary and E. N. Johnson, “Theory and flight-test validation of
a concurrent-learning adaptive controller,” Journal of Guidance, Control,
and Dynamics, vol. 34, no. 2, pp. 592–607, 2011.
[38] G. Chowdhary, Concurrent learning for convergence in adaptive control
without persistency of excitation. PhD thesis, Georgia Institute of
Technology, 2010.
[39] S. Kersting and M. Buss, “Concurrent learning adaptive identification
of piecewise affine systems,” in Conference on Decision and Control,
pp. 3930–3935, IEEE, 2014.
[40] M. Krstic, P. V. Kokotovic, and I. Kanellakopoulos, Nonlinear and
adaptive control design. John Wiley & Sons, Inc., 1995.
[41] E. Lavretsky, T. E. Gibson, and A. M. Annaswamy, “Projection operator
in adaptive systems,” arXiv preprint arXiv:1112.4232, 2011.
[42] H. K. Khalil, “Nonlinear systems, 3rd,” New Jewsey, Prentice Hall,
vol. 9, 2002.

