Adaptive tracking control for non-periodic reference signals under quantized observations
Chuiliu Kong Ying Wang




1]Chuiliu Kong

2]Ying Wang



KONG et al


[1]Research Center for Mathematics and Interdisciplinary Sciences, Shandong University, Qingdao, Shandong Province, 266237, China

[2]Key Laboratory of Systems and Control, Institute of Systems Science, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China



*Ying Wang,  wangying96@amss.ac.cn



[Summary]	This paper considers an adaptive tracking control problem for stochastic regression systems with multi-threshold quantized observations. Different from the existing studies for periodic reference signals, the reference signal in this paper is non-periodic. Its main difficulty is how to ensure that the designed controller satisfies the uniformly bounded and excitation conditions that guarantee the convergence of the estimation in the controller under non-periodic signal conditions. This paper designs two backward-shifted polynomials with time-varying parameters and a special projection structure, which break through periodic limitations and establish the convergence and tracking properties. To be specific, the adaptive tracking control law can achieve asymptotically optimal tracking for the non-periodic reference signal; Besides, the proposed estimation algorithm is proved to converge to the true values in almost sure and mean square sense, and the convergence speed can reach O(1/k) under suitable conditions. Finally, the effectiveness of the proposed adaptive tracking control scheme is verified through a simulation.














Adaptive tracking control for non-periodic reference signals under quantized observations
    [
    
=========================================================================================








§ INTRODUCTION


Since the actual control systems usually suffer from uncertainties stemming from parameters, structure, and environment (e.g., system aging, component failures, and external disturbances), the development of adaptive control theory provides a method to deal with the above uncertainties. In addition, owing to the limited communication ability and computation ability of the sensors, the outputs of the actual control systems are often with finite precision, i.e., it is only known whether the output belongs to a set, not the exact value (e.g., <cit.>,  <cit.> and <cit.>). Therefore, considering the actual problem requirements, the quantized adaptive control techniques provide an effective approach to control problems involving quantized measurements.

Usually, the design of adaptive control is based on the estimation of unknown system parameters to obtain the desired system performance. For the difference of the unknown parameters to be estimated in the controller, the design of adaptive control is subdivided into direct adaptive method and indirect adaptive method. In detail, in the direct adaptive method (e.g., <cit.>, <cit.> and <cit.>), we directly estimate the unknown parameters included in the designed controller. Whereas in the indirect adaptive method (e.g., <cit.>-<cit.>,  and <cit.>-<cit.>, <cit.>, <cit.>), the original unknown parameters in the system are directly estimated and then the adaptive control is designed by the above estimations. This means that the parameters required in the former method do not necessarily have to estimate the true value of the unknown parameter in the system, whereas that true value is required in the latter method. For example, in the framework of the direct adaptive control, Zhang et al. <cit.> proposed an implicit function-based adaptive control scheme for the discrete-time neural-network systems and nonlinear systems in a general non-canonical form, respectively. Wen et al. <cit.> studied the convergence of the output tracking error for nonlinear adaptive control systems. It should be noted that they consider deterministic systems. Moreover, in the framework of the indirect adaptive, there are many classic methodologies about stochastic adaptive control such as stochastic gradient algorithm-based adaptive control method <cit.> and least-squares algorithm-based self-tuning regulator <cit.>, and so on. Numerous adaptive control theories have been derived from the aforementioned methods and techniques, such as <cit.>. In this paper, the tracking problem is studied in the indirect adaptive framework. The reason is that on the one hand, the original unknown parameters in the system can be estimated accurately, on the other hand, the designed adaptive control based on the estimations of the unknown parameters is more robust.

It is worth noting that the aforementioned methodologies are constructed based on the accurate information of system output, or output with measurement noises. However, as mentioned above, obtaining the exact outputs of the systems may be not achievable, and in some cases only quantized measurements of the system outputs can be obtained. These quantized measurements inherently contain measurement uncertainty about the system outputs. Therefore, the research goal of this paper is how to use the quantized observations to design an adaptive control law to achieve accurate control of the systems, which falls into the category of robust control. The existing research on adaptive control with quantized measurements has been somewhat studied but is still relatively lacking. For example, for first-order stochastic regression systems with binary-valued observations, Guo et al. in <cit.> designed an adaptive tracking control and finally achieved the asymptotically optimal tracking for deterministic reference signal sequence. Further, for the high-order stochastic regression systems with binary-valued observations, Zhao et al. in <cit.> proposed a two-scale algorithm and achieved the asymptotically optimal tracking for the periodic reference signal. In this algorithm, on the one hand, the designed adaptive control is required to last at a long time scale. On the other hand, the unknown parameter is estimated by the empirical measurement method <cit.> at a small time scale, which is an offline identification algorithm. Since adaptive control in the above two-scale algorithm is limited to remain constant over time, it results in a possible reduction of the tracking speed. Therefore, to further optimize the tracking speed, with the help of an online recursive projection identification algorithm in <cit.>, Wang et al. in <cit.> designed another adaptive tracking control law without waiting time and finally achieved the desired tracking effect for the periodic reference signal. For the case of system outputs that are quantized and also contain observation uncertainty, Kong et al. in <cit.> investigated an adaptive tracking control problem for periodic reference signals and obtained the effect of observation uncertainty on the designed algorithm. Furthermore, a detailed description of the quantized system identification theory can be referred to <cit.>.


At present, by summarizing the existing research as mentioned above, it is easy to see that there are two kinds of shortcomings in the tracking control problem for stochastic regression systems containing quantized observations: one is that the system model studied is relatively simple, that is, the system model is first-order <cit.>, the other is that the reference signal is limited to periodicity <cit.>. The reason for considering the quantized tracking control problem in the above two types of frameworks is to facilitate the solution of the following two problems, which are precisely the ones that need to be overcome in the framework of this paper, i.e.,

∙ The adaptive tracking control design is coupled with the parameter estimation algorithm <cit.>; 

∙ Losing the periodicity condition, in contrast to  <cit.>, this paper faces difficulties in ensuring that controllers designed in accordance with the certainty equivalent principle satisfy both persistent excitation and uniformly bounded conditions.

Therefore, this paper is concerned with the quantized tracking control problem where the stochastic regression system is two-order and the reference signal is non-periodic at the same time, whose model is relatively more general than the first-order quantized system in <cit.> and reference signals are more general than the periodic case in <cit.>. This paper designs two backward-shifted polynomials with time-varying parameters and a special projection structure to overcome the aforementioned two difficulties. However, it also should be noted that this control scheme cannot be directly used to handle the quantized system with higher-order cases. It is because, for higher-order quantized systems, there remains some difficulty in ensuring that adaptive tracking control law designed by the certainty equivalence principle satisfies the uniformly bounded condition. Actually, this paper is the first attempt to address the tracking problem for non-periodic reference signals under higher-order quantized systems, and provides referential ideas for further solving the tracking control problem under more general quantized system models. The main contributions of this paper can be summarized as follows.




i) Model and Problem aspects: On the one hand, the outputs of the stochastic regression model considered in this paper are quantized by a multi-threshold quantizer, which introduces a strong non-linearity to the system. On the other hand, the adaptive tracking control problem considered in this paper is for the non-periodic reference signals. This research breaks through the limitations of previous literature on first-order system model <cit.> and periodic reference signals (e.g., <cit.>, <cit.> and <cit.>). It makes the system models and the research results studied in this paper to be extended to a wider scope.




ii) Method aspect: In this paper, we design two backward-shifted polynomials with time-varying parameters and a special projection structure to make the designed adaptive tracking control law satisfy the persistent excitation and uniformly bounded conditions, which are used to guarantee the convergence properties of the estimation algorithm. Compared with <cit.>, the system outputs considered in this paper are quantized by a multi-threshold quantizer.




iii) Result aspect: The adaptive tracking control scheme designed in this paper not only ensures that the estimations of the unknown parameters converge to the true values but most importantly also achieves the asymptotically optimal tracking effect for the non-periodic reference signals. In addition, the mean square convergence speed for the unknown parameter estimation can also reach O(1/k) as that in <cit.>.




The rest of this article is organized as follows. In Section 2, we introduce the tracking control problem with multi-threshold quantized outputs. In Section 3, an adaptive tracking control scheme is presented. In Section 4, the main results including the convergence, convergence speed, and the performance of adaptive tracking control are given. In Section 5, a simulation is given to verify the above main conclusions. Finally, we summarize the conclusions of this article in Section 6.

Notations.
First of all, we define some notations. Let · be the Euclidean norm, ⌊ x⌋ ={a∈ℤ|a≤ x} and ⌈ x⌉={a∈ℤ|a≥ x} be the largest integer less than or equal to x and the smallest integer large than or equal to x for any x∈ℝ, respectively. For two real number sequence {a_1,k, k≥ 1} and {a_2,k, k≥ 1}, one writes a_1,k=O(a_2,k) if and only if there exist a positive real number M_0 and an integer k_0 such that |a_1,k|≤ M_0|a_2,k| for all k≥ k_0.



§ PROBLEM FORMULATION

In this section, we consider the following stochastic regression model:

    y(k)=ϕ^⊺(k)θ+w(k), k=1,2,...

where ϕ(k)▵=[u(k),u(k-1)]^⊺∈ℝ^2, u(s)=0, for s≤0, θ▵=[θ(1),θ(2)]^⊺∈ℝ^2, and {w(k)}_k≥1 are the control input, unknown parameter and system noise, respectively. y(k)∈ℝ is the measured output, which can not be exactly measured and only its quantized observation can be obtained by a sensor with multi-threshold C_p, for p=0,1,...,m, and C_0<C_1<C_2<⋯<C_m<C_m+1. Then, the quantized observation is



    S(k)=∑_p=0^mI_{C_p< y(k)≤ C_p+1},



where C_0=-∞, C_m+1=+∞ and the indicator function I_{C_p< y(k)≤ C_p+1} is defined by

    I_{C_p< y(k)≤ C_p+1}=
    	{   1,   y(k)∈(C_p,C_p+1],
       0,  .
    	.



Next, some necessary assumptions are introduced. Firstly, set {y^*(k)}_k≥1 as the known reference signal, which satisfies the following two assumptions.


	The reference signal {y^*(k)}_k≥1 is a bounded sequence, i.e., |y^*(k)|≤y̅^*, where y̅^* is a given positive constant.


	Denote 𝒴(i)=[y^*(i),y^*(i-1)]^⊺. There exist constants h> 2 and δ_y>0 such that
	
    λ_min(∑_i=k+1^k+h-1𝒴(i)𝒴^⊺(i))>δ_y, k=1,2,...




	The purpose of this paper is to design an adaptive tracking control {u(k)}_k≥ 1 to drive the output {y(k)}_k≥1 to follow the non-periodic reference signal {y^*(k)}_k≥1. Therefore, the model and problem considered in this paper are fundamentally different from the previous literature on first-order system model <cit.> and periodic reference signals (e.g., <cit.>, <cit.> and <cit.>). 


Secondly, in order to proceed with our analysis, some other assumptions about the unknown parameter and noises are introduced as follows.

	There exists a constant μ∈(0,1) such that |Q(ω)|>μ for |ω|≤ 1 with Q(ω)=θ(1)+θ(2)ω. 



	If μ=0, then Assumption <ref> turns to be the minimum phase condition, which is necessary for adaptive
	tracking problem. Moreover, Assumption <ref> implies that the unknown parameter θ(1)≠0 and |θ(1)/θ(2)|>1.



	Let Θ denote the set of parameters satisfy Assumption <ref>. The parameter θ belongs to a convex compact set Ω and Ω is the subset of Θ, i.e., θ∈Ω⊆Θ. Besides, M̅=sup_v∈Ωv, 0<θ≤|θ(1)| and |θ(2)|<θ̅, θ and θ̅ are constants and θ>θ̅>0.



	Let us give an example of Assumptions <ref>-<ref>. For example, let ω▵=θI_{sgn(θ(1))>0}-M̅I_{sgn(θ(1))<0} and ω̅▵=M̅I_{sgn(θ(1))>0}-θI_{sgn(θ(1))<0}, if the sign of the unknown parameter θ(1) is known, then the set Ω in Assumption <ref> can be constructed as {(ω(1),ω(2))|ω≤ω(1)≤ω̅, |ω(2)|≤θ̅}, where the positive constants θ̅, θ, M̅ satisfy θ̅<θ≤M̅. 
	
	



	{w(k), k≥ 1} is an i.i.d. random sequence with zero mean and finite variance. The probability distribution function F(·) of w(1) is assumed to be known. 


Then, based on the above system (<ref>)-(<ref>), the tracking control problem considered in this paper is detailed as follows:

	Let {y^*(k)}_k≥1 be a known non-periodic reference signal and satisfy Assumptions <ref> and <ref>. The aim of this paper is to construct an adaptive control ϕ(k) based on the quantized observations {S(1),S(2),...,S(k-1)} and the past inputs {u(1),u(2),...,u(k-1)} to minimize the following index:
	
    J(k)=𝔼[(y(k)-y^*(k))^2], ∀ k≥ 1.





§ ADAPTIVE CONTROL ALGORITHM

Firstly,  we consider the design of the controller u(k). To be specific, when the parameter θ is known, the optimal input ϕ̅(k)▵=[u̅(k),u̅(k-1)]^⊺ can be designed as:

    u̅(k)=1/θ(1)y^*(k)-θ(2)/θ(1)u̅(k-1), k=1,2,...

Secondly, when the parameter θ is unknown, it is first necessary to estimate the unknown parameter and then design the corresponding control input by the certainty equivalence principle. Therefore, for system  (<ref>)-(<ref>) and index (<ref>), inspired by the recursive projection identification algorithm in <cit.>, the adaptive control input can be designed as the following steps.

Step 1.(Initiation): Let θ̂(0)▵=[θ̂(1,0),θ̂(2,0)]^⊺∈Ω, ϕ(1)=[u(1),0]^⊺, where u(1)=y^*(1)/θ̂(1,0).

Step 2.(Weighted conversion of quantized observations): Based on S(k) in (<ref>), let S̅(k)=∑_p=0^mβ_p I_{C_p<y(k)≤ C_p+1}, where β_p is a quantized weight with respect to set (C_p,C_p+1] for p=0,1,...,m, which satisfies β_0>β_1>⋯>β_m and can be designed. 

Step 3.(Identification): The online stochastic approximation-type identification algorithm is as follows:

    {θ̂(k)=   Π_Ω(θ̂(k-1)+ϕ(k)/k(A(k)-S̅(k))),
    
    		A(k)=   (∑_p=0^mβ_p[F(C_p+1-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ̂(k-1))]),
    	.

where θ̂(k)▵=[θ̂(1,k),θ̂(2,k)]^⊺ is the estimation of θ at time k, Π_Ω(x)=_κ∈Ωx-κ, ∀ x∈ℝ^2 is the projection operator. 

Step 4.(Design of control input): By the certainty equivalent principle, the optimal input ϕ(k+1)=[u(k+1),u(k)]^⊺ in the case of unknown parameters is designed as follows:

    u(k+1)=1/θ̂(1,k)y^*(k+1)-θ̂(2,k)/θ̂(1,k)u(k), k=1,2,...


Step 5.(Update): Let k=k+1, the return to Step 2.



§ MAIN RESULTS

In this section, the convergence of the estimation algorithm and the asymptotic validity of the adaptive tracking control are given. Firstly, we give some results about the projection operator and the input ϕ(k).


	The projection operator Π_Ω(·) satisfies
	
    Π_Ω(x_1)-Π_Ω(x_2)≤x_1-x_2,  ∀ x_1, x_2∈ℝ^2.







	For the input ϕ(k) designed by (<ref>), under Assumptions <ref>-<ref>, we have the following two assertions.

	i)The uniformly bounded condition holds, i.e., there exists a positive constant M such that
	
    sup_k≥ 1ϕ(k)≤ M.

	ii)The following persistent excitation condition holds.
	
    λ_min(∑_i=k+2^k+h+1ϕ(i)ϕ^⊺(i))>δ,  k≥ K_0,

	where δ▵=δ_y/4(h-1)M̅^2, δ_y, h and M̅ are defined in Assumptions <ref> and <ref>, respectively.



	
	i) By (<ref>), the control input u(k) can be expressed as
	
    u(k)=y^*(k)/θ̂(1,k-1)+∑_i=1^k-1(-1)^k-i∏_j=i^k-1θ̂(2,j)/∏_j=i-1^k-1θ̂(1,j)y^*(i).

	Hence, for ∀ k≥ 1, we have
	
    |u(k)|≤y̅^*/θ+∑_i=1^k-1θ̅^k-i/θ^k-i+1y̅^*≤y̅^*/θ-θ̅.

	Denote M=√(2)y̅^*/θ-θ̅. It results in sup_k≥ 1ϕ(k)≤ M.
	
	ii) Denote A_k(z)=1+θ̂(2,k-1)/θ̂(1,k-1)z and B_k(z)=1/θ̂(1,k-1), where z is the shift-back operator. Therefore, the control input (<ref>) can be written as
	
    A_k(z)u(k)=B_k(z)y^*(k).

	Let ψ(k)=A_k(z)ϕ(k)=[B_k(z)y^*(k),B_k-1(z)y^*(k-1)+Δ_k-2^k-1u(k-2)]^⊺, where
	

	
    Δ_k-2^k-1▵=θ̂(2,k-1)/θ̂(1,k-1)-θ̂(2,k-2)/θ̂(1,k-2).

	

	On the one hand, for any x∈ℝ^2, we obtain
	
    x^⊺[∑_i=k+3^k+h+1ψ(i)ψ^⊺(i)]x
    			=   ∑_i=k+3^k+h+1[x^⊺(ϕ(i)+θ̂(2,i-1)/θ̂(1,i-1)ϕ(i-1))]^2
    ≤   2max{1,θ̅^2/θ^2}[∑_i=k+3^k+h+1∑_j=1^2[x^⊺ϕ(i-j+1)]^2]
    ≤    2(h-1)[∑_i=k+2^k+h+1[x^⊺ϕ(i)ϕ^⊺(i)x]].

	

	Hence,
	
    2(h-1)λ_min(∑_i=k+2^k+h+1ϕ(i)ϕ^⊺(i))≥λ_min(∑_i=k+3^k+h+1ψ(i)ψ^⊺(i)).

	On the other hand, we just need to prove the following inequality holds.
	
    λ_min(∑_i=k+3^k+h+1ψ(i)ψ^⊺(i))≥ Kλ_min(∑_i=k+3^k+h+1𝒴(i)𝒴^⊺(i)),  K>0.

	Define the following two backward-shifted polynomials with time-varying parameters
	

	
    H_x,k(z)=x(1)B_k(z)+x(2)B_k-1(z)z=∑_l=0^1g_l,k(x)z^l, Δ_x,k(z)=x(2)Δ_k-2^k-1z^2,

	where x=[x(1),x(2)]^⊺ and g_k(x)=[g_0,k(x),g_1,k(x)]^⊺=[x(1)/θ̂(1,k-1),x(2)/θ̂(1,k-2)]^⊺.
	
	Firstly, by (<ref>), we have x^⊺ψ(i)=H_x,i(z)y^*(i)+Δ_x,i(z)u(i). Then, we have
	
    ∑_i=k+3^k+h+1(x^⊺ψ(i))^2
    			=   ∑_i=k+3^k+h+1[H_x,i(z)y^*(i)+x(2)Δ_i-2^i-1u(i-2)]^2
    ≥   2[∑_i=k+3^k+h+1H_x,i(z)y^*(i)x(2)Δ_i-2^i-1u(i-2)  ]+∑_i=k+3^k+h+1[∑_p=0^1g_p,i(x)y^*(i-p)]^2
    
    			=   ∑_i=k+3^k+h+1[g^⊺_i(x)𝒴(i)]^2+2[∑_i=k+3^k+h+1H_x,i(z)y^*(i)x(2)Δ_i-2^i-1u(i-2)  ]
    ≥   2[∑_i=k+3^k+h+1g^⊺_k+2(x)𝒴(i)𝒴^⊺(i)(g_i(x)-g_k+2(x)) ]+2[∑_i=k+3^k+h+1H_x,i(z)y^*(i)x(2)Δ_i-2^i-1u(i-2)  ]
       +g^⊺_k+2(x)∑_i=k+3^k+h+1[𝒴(i)𝒴^⊺(i)]g_k+2(x).

We just need to deal with the last two terms of (<ref>) separately. Since
	
    g_k+2(x)≤√((x(1))^2/θ^2+(x(2))^2/θ^2)=x/θ, 𝒴(i)≤√(2)y̅^*,

	then, denote K_1=2(y̅^*)^2x/θ, the following relation holds.
	
    ∑_i=k+3^k+h+1|g^⊺_k+2(x)𝒴(i)𝒴^⊺(i)(g_i(x)-g_k+2(x))|
    ≤   [∑_i=k+3^k+h+1g^⊺_k+2(x)×𝒴(i)×𝒴^⊺(i)×(g_i(x)-g_k+2(x))]
    ≤   K_1	[∑_i=k+3^k+h+1√((x(1)/θ̂(1,i-1)-x(1)/θ̂(1,k+1))^2+(x(2)/θ̂(1,i-2)-x(2)/θ̂(1,k))^2)]
    
    			=   K_1	[∑_i=k+3^k+h+1√((x(1)(θ̂(1,k+1)-θ̂(1,i-1))/θ̂(1,i-1)θ̂(1,k+1))^2+(x(2)(θ̂(1,k)-θ̂(1,i-2))/θ̂(1,i-2)θ̂(1,k))^2)]
    
    			≤   K_1√(max_i=1,2{x^2(i)})/θ^2[∑_i=k+3^k+h+1√(∑_j=1^2θ̂(k+2-j)-θ̂(i-j)^2)].

	Denote K_2=2(y̅^*)^2x√(max_i=1,2{x^2(i)})/θ^3. Since ||θ̂(i)-θ̂(j)||^2≤M^2(i-j)^2/(j+1)^2(m∑_p=0^mβ^2_p), j≤ i, then we have
	
    ∑_i=k+3^k+h+1|g^⊺_k+2(x)𝒴(i)𝒴^⊺(i)(g_i(x)-g_k+2(x))|≤    K_2[∑_i=k+3^k+h+1√(∑_j=1^2M^2(i-k-2)^2/(k+3-j)^2m∑_p=0^mβ^2_p)]
    ≤   K_2M(h-1)^3/2√(2m∑_p=0^mβ^2_p)/k+2	=O(1/k).

	Next, we estimate the last term of (<ref>). 
	
    ∑_i=k+3^k+h+1| H_x,i(z)y^*(i)x(2)Δ_i-2^i-1u(i-2)|
    ≤   My̅^*(|x(1)+|x(2)|)|x(2)|/θ[∑_i=k+3^k+h+1|Δ_i-2^i-1| ]
    
    			=   My̅^*(|x(1)+|x(2)|)|x(2)|/θ∑_i=k+3^k+h+1[|θ̂(2,i-1)/θ̂(1,i-1)-θ̂(2,i-2)/θ̂(1,i-2)|]
    
    			=   My̅^*(|x(1)+|x(2)|)|x(2)|/θ∑_i=k+3^k+h+1[|θ̂(1,i-2)θ̂(2,i-1)-θ̂(1,i-1)θ̂(2,i-2)|/|θ̂(1,i-1)θ̂(1,i-2)|]
    ≤   My̅^*(|x(1)+|x(2)|)|x(2)|/θ^3∑_i=k+3^k+h+1[|θ̂(1,i-2)||θ̂(2,i-1)-θ̂(2,i-2)|+|θ̂(2,i-2)||θ̂(1,i-1)-θ̂(1,i-2)| ]
    ≤   √(2)MM̅y̅^*(|x(1)+|x(2)|)|x(2)|/θ^3∑_i=k+3^k+h+1[θ̂(i-1)-θ̂(i-2)].

	Since ||θ̂(i)-θ̂(j)||≤M(i-j)/(j+1)∑_p=0^m|β_p|, j≤ i, then
	
    ∑_i=k+3^k+h+1| H_x,i(z)y^*(i)x(2)Δ_i-2^i-1u(i-2)|≤   √(2)MM̅y̅^*(|x(1)+|x(2)|)|x(2)|/θ^3∑_i=k+3^k+h+1M/i-1∑_p=0^m|β_p|
    ≤   √(2)(h-1)M^2M̅y̅^*(|x(1)+|x(2)|)|x(2)|/θ^3(k+2)∑_p=0^m|β_p|
    
    			=   O(1/k).

	Finally, let x be the unit vector. Then g_k(x)^2=∑_i=1^2x^2(i)/θ̂^2(1,k-i)∈[1/M̅^2, 1/θ^2]. Taking (<ref>)-(<ref>) into (<ref>), we obtain the following relations.
	
    x^⊺[∑_i=k+3^k+h+1ψ(i)ψ^⊺(i)]x≥   g^⊺_k+2(x)∑_i=k+3^k+h+1[𝒴(i)𝒴^⊺(i)]g_k+2(x)-O(1/k)
    ≥   λ_min(∑_i=k+3^k+h+1𝒴(i)𝒴^⊺(i))/M̅^2-O(1/k).

	Therefore, there exists K_0>0 such that for k≥ K_0,
	
    x^⊺[∑_i=k+3^k+h+1ψ(i)ψ^⊺(i)]x≥λ_min(∑_i=k+3^k+h+1𝒴(i)𝒴^⊺(i))/2M̅^2.

	Therefore, based on (<ref>), we obtain
	
    λ_min(∑_i=k+3^k+h+1ψ(i)ψ^⊺(i))≥δ_y/2M̅^2,   k≥ K_0.

	Therefore, according to (<ref>) and (<ref>), we obtain
	
    λ_min(∑_i=k+2^k+h+1ϕ(i)ϕ^⊺(i))≥δ_y/4(h-1)M̅^2,   k≥ K_0.

	This completes the proof.



	For any given ζ∈ℝ, γ∈ℝ and r∈ℤ^+, we have the following assertions as k→∞:
	
    ∏_i=r^k(1-ζ/i)=O(1/k^ζ),

	and
	
    ∑_l=r^k-1∏_i=l+1^k(1-ζ/i)1/l^γ=
    		{   O(1/k^γ-1),    γ<ζ+1,
       O(ln (k)/k^ζ),     γ=ζ+1,
       O(1/k^ζ),    γ>ζ+1.
    		.



In addition, for system noise {w(k)}_k≥ 1, we need the following additional assumption in the framework of this section.

	The probability density function of w(1) denoted by f(·). Besides, there exists a minimum f^* of f(·) on interval [-D_1,D_1], where D_1=max{|C_m|,|C_1|}+MM̅.	


Denote the estimation error θ̃(k)=θ̂(k)-θ. Then, the following results give the convergence property of the estimation error θ̃(k).

	Under Assumptions <ref>-<ref>, the mean square convergence rate of the estimation error is given as follows:
	
    𝔼[θ̃(k)^2]=
    		{   O(1/k),     2(β_0-β_m)δf^*/h>1;
       O(ln (k)/k),     2(β_0-β_m)δf^*/h=1;
       O(1/k^2(β_0-β_m)δf^*/h),     2(β_0-β_m)δf^*/h<1,
    		.

	where h, f^* and δ are defined in Assumptions <ref>, <ref> and Proposition <ref>, respectively.


	By (<ref>) and Proposition <ref>, we have
	
    θ̃(k)^2=   Π_Ω(θ̂(k-1)+ϕ(k)/k(A(k)-S̅(k)))-Π_Ω(θ)^2
    ≤   θ̃(k-1)+ϕ(k)/k(A(k)-S̅(k))^2
    ≤   θ̃(k-1)+ϕ(k)/k(∑_p=0^mβ_p[F(C_p+1-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ̂(k-1))]-S̅(k))^2
    
    			=   θ̃(k-1)^2+ϕ(k)^2/k^2(∑_p=0^mβ_p[F(C_p+1-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ̂(k-1))]-S̅(k))^2
       +2θ̃^⊺(k-1)ϕ(k)/k(∑_p=0^mβ_p[F(C_p+1-ϕ^⊺(k)θ̂(
    			k-1))-F(C_p-ϕ^⊺(k)θ̂(k-1))]-S̅(k)).

	
	Denote ℱ_k-1=σ{w(s), s≤ k-1}. By (<ref>)-(<ref>), we obtain θ̃(k-1) and ϕ(k) are ℱ_k-1-measurable. Taking conditional expectation with respect to S̅(k), we obtain
	
    𝔼[S̅(k)|ℱ_k-1]=   𝔼[∑_p=0^mβ_pI_{C_p<y(k)≤ C_p+1}|ℱ_k-1]
    
    			=   𝔼[∑_p=0^mβ_pI_{C_p-ϕ^⊺(k)θ<w(k)≤ C_p+1-ϕ^⊺(k)θ}|ℱ_k-1]
    
    			=   ∑_p=0^mβ_p[F(C_p+1-ϕ^⊺(k)θ)-F(C_p-ϕ^⊺(k)θ)].
	
	Therefore, 	based on (<ref>), we have
	
    𝔼[2θ̃^⊺(k-1)ϕ(k)/k(∑_p=0^mβ_p[F(C_p+1-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ̂(k-1))]-S̅(k))|ℱ_k-1]
    
    			
    			
    			=   2θ̃^⊺(k-1)ϕ(k)/k[∑_p=0^m-1β_p[F(C_p+1-ϕ^⊺(k)θ̂(k-1))-F(C_p+1-ϕ^⊺(k)θ)]-∑_p=1^mβ_p[F(C_p-ϕ^⊺(k)θ̂(k-1))
       -F(C_p-ϕ^⊺(k)θ)]]
    
    			=   2θ̃^⊺(k-1)ϕ(k)/k[∑_p=1^mβ_p-1[F(C_p-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ)]-∑_p=1^mβ_p[F(C_p-ϕ^⊺(k)θ̂(k-1))
       -F(C_p-ϕ^⊺(k)θ)]]
    
    			=   2θ̃^⊺(k-1)ϕ(k)/k∑_p=1^m(β_p-1-β_p)[F(C_p-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ)].

	According to the mean value theorem, there exists η_p(k)∈(C_p-ϕ^⊺(k)θ̂(k-1),C_p-ϕ^⊺(k)θ) or (C_p-ϕ^⊺(k)θ,C_p-ϕ^⊺(k)θ̂(k-1)) such that
	
    F(C_p-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ)=-f(η_p(k))ϕ^⊺(k)θ̃(k-1).

	
	Thus, based on (<ref>) and Assumption <ref>, (<ref>) can be rewritten as
	
    𝔼[2θ̃^⊺(k-1)ϕ(k)/k(∑_p=0^mβ_p[F(C_p+1-ϕ^⊺(k)θ̂(k-1))-F(C_p-ϕ^⊺(k)θ̂(k-1))]-S̅(k))|ℱ_k-1]
    
    			=   2θ̃^⊺(k-1)ϕ(k)/k∑_p=1^m(β_p-1-β_p)[-f(η_p(k))ϕ^⊺(k)θ̃(k-1)]
    ≤   -2(β_0-β_m)f^*θ̃^⊺(k-1)ϕ(k)ϕ^⊺(k)θ̃(k-1)/k.

	By (<ref>), (<ref>) can be written as
	
    𝔼[θ̃(k)^2]≤𝔼[θ̃(k-1)^2]+M^2(m∑_p=0^mβ^2_p)/k^2-2(β_0-β_m) f^*/k𝔼[θ̃^⊺(k-1)ϕ(k)ϕ^⊺(k)θ̃(k-1)].

	
	Consequently, we expand θ̃(k)^2 to θ̃(k-h)^2 by (<ref>) and obtain
	
    𝔼[θ̃(k)^2]≤𝔼[θ̃(k-h)^2]+∑_i=k-h+1^kM^2(m∑_p=0^mβ^2_p)/i^2-∑_i=k-h+1^k2(β_0-β_m) f^*/i𝔼[θ̃^⊺(i-1)ϕ(i)ϕ^⊺(i)θ̃(i-1)].

	Since for i=k-h+1,...,k, we have
	
    θ̃^⊺(i-1)ϕ(i)ϕ^⊺(i)θ̃(i-1)
    
    			=   (θ̃(i-1)-θ̃(k-h)+θ̃(k-h))^⊺ϕ(i)ϕ^⊺(i)(θ̃(i-1)-θ̃(k-h)+θ̃(k-h))
    
    			=   θ̃(k-h)^⊺ϕ(i)ϕ^⊺(i)θ̃(k-h)+(θ̃(i-1)-θ̃(k-h))^⊺ϕ(i)ϕ^⊺(i)(θ̃(i-1)-θ̃(k-h))
       +2(θ̃(i-1)-θ̃(k-h))^⊺ϕ(i)ϕ^⊺(i)θ̃(k-h),

	and Proposition <ref>, then we have
	
    -∑_i=k-h+1^k1/i𝔼[θ̃^⊺(i-1)ϕ(i)ϕ^⊺(i)θ̃(i-1)]
    
    			=   -∑_i=k-h+1^k1/i𝔼[θ̃(k-h)^⊺ϕ(i)ϕ^⊺(i)θ̃(k-h)]-∑_i=k-h+1^k2/i𝔼[(θ̃(i-1)-θ̃(k-h))^⊺ϕ(i)ϕ^⊺(i)θ̃(k-h)]
       -∑_i=k-h+1^k1/i𝔼[(θ̃(i-1)-θ̃(k-h))^⊺ϕ(i)ϕ^⊺(i)(θ̃(i-1)-θ̃(k-h))]
    ≤   -∑_i=k-h+1^k1/i𝔼[θ̃(k-h)^⊺ϕ(i)ϕ^⊺(i)θ̃(k-h)]-∑_i=k-h+1^k2/i𝔼[(θ̃(i-1)-θ̃(k-h))^⊺ϕ(i)ϕ^⊺(i)θ̃(k-h)]
    ≤   2M̅M^2/k-h+1∑_i=k-h+1^k√(𝔼[θ̃(i-1)-θ̃(k-h)^2])-δ/k𝔼[θ̃(k-h)^2]
    ≤   -δ/k𝔼[θ̃(k-h)^2]+2M̅(h-1)h M^3/(k-h)^2√(m∑_p=0^mβ^2_p).

	Therefore, by (<ref>), (<ref>) can be expressed as
	
    𝔼[θ̃(k)^2]≤   (1-2δ(β_0-β_m) f^*/k)𝔼[θ̃(k-h)^2]+hM^2/(k-h+1)^2(m∑_p=0^mβ^2_p)
       +4M̅(h-1)hM^3(β_0-β_m)f^* M^3/(k-h)^2√(m∑_p=0^mβ^2_p)
    ≤   (1-2δ(β_0-β_m) f^*/k)𝔼[θ̃(k-h)^2]+O(1/(k-h)^2).

	According to the inequality (<ref>), expanding θ̃(k)^2 to θ̃(k-⌊k/h⌋h)^2 yields
	
	
    𝔼[θ̃(k)^2]≤   ∏^⌊k/h⌋-1_j=0(1-2δ(β_0-β_m) f^*/k-jh)𝔼[θ̃(k-⌊k/h⌋h)^2]
       +∑_j=2^⌊k/h⌋∏_i=0^j-2(1-2δ(β_0-β_m) f^*/k-ih)O(1/(k-jh+1)^2)+O(1/(k-h)^2).

	Firstly, the first item of the right side of (<ref>) is
	
    ∏^⌊k/h⌋-1_j=0(1-2δ(β_0-β_m) f^*/k-jh)≤∏^⌈k/h⌉_m=⌈k/h⌉-⌊k/h⌋+1(1-2δ(β_0-β_m) f^*/mh).

Secondly, the second item of the right side of (<ref>) is
	
    ∑_j=2^⌊k/h⌋∏_i=0^j-2(1-2δ(β_0-β_m) f^*/k-ih)O(1/(k-jh+1)^2)
    ≤   ∑_j=2^⌊k/h⌋∏_i=0^j-2(1-2δ(β_0-β_m) f^*/k-ih)O(1/(⌊k/h⌋-j+1/h)^2h^2)
    ≤   ∑_m=1^⌊k/h⌋-2∏^⌈k/h⌉_p=⌈k/h⌉-⌊k/h⌋+m+2(1-2δ(β_0-β_m) f^*/ph)O(1/m^2h^2)+O(∏_i=0^⌊k/h⌋-2(1-2δ(β_0-β_m)f^*/k-ih)).

	Thus, we have
		
    𝔼[θ̃(k)^2]≤   ∏^⌈k/h⌉_m=⌈k/h⌉-⌊k/h⌋+1(1-2δ(β_0-β_m) f^*/mh)𝔼[θ̃(k-⌊k/h⌋h)^2]
       +∑_m=1^⌊k/h⌋-2∏^⌈k/h⌉_p=⌈k/h⌉-⌊k/h⌋+m+2(1-2δ(β_0-β_m) f^*/ph)O(1/m^2h^2)
       +O(∏_i=0^⌊k/h⌋-2(1-2δ(β_0-β_m) f^*/k-ih))+O(1/(k-h)^2).

	
	Therefore, by Lemma <ref>, we finally obtain
		
    𝔼[θ̃(k)^2]=
    			{   O(1/k),     2(β_0-β_m)δf^*/h>1;
       O(ln (k)/k),     2(β_0-β_m)δf^*/h=1;
       O(1/k^2(β_0-β_m)δf^*/h),     2(β_0-β_m)δf^*/h<1,
    			.

	This completes the proof.



	Under Assumptions <ref>-<ref>, the estimation in (<ref>) will convergence to the real parameter θ in the sense of mean square and almost surely
	
    lim_k→∞𝔼[θ̃(k)^2]=0,  lim_k→∞θ̂(k)=θ, a.s.



	By Theorem <ref>, it reduces that 
	
    lim_k→∞𝔼[θ̃(k)^2]=0.

	Since ϕ(k) and θ̂(k-1) are ℱ_k-1-measurable, we have by (<ref>)
	
    𝔼[θ̃(k)^2|ℱ_k-1]≤θ̃(k-1)^2+M^2(m∑_p=0^mβ_p^2)/k^2.

	Also
	
    ∑_k=1^∞M^2(m∑_p=0^mβ_p^2)/k^2<∞.

	Together with Lemma 1.2.2 in <cit.>, it results in that θ̃(k) almost surely converges to a bounded limit. Notice that lim_k→∞𝔼[θ̃(k)^2]=0. Then, there exists a subsequence of θ̃(k) that almost surely converges to 0. Consequently, θ̃(k) almost surely converges to 0, i.e.,
	
    lim_k→∞θ̂(k)=θ, a.s.

	This completes the proof.


Finally, we give the following result about the closed-loop system consisting of (<ref>) and (<ref>).


	Under Assumptions <ref>-<ref>, the adaptive control law (<ref>) is asymptotically optimal, i.e.,
	
    lim_k→∞𝔼[(y(k)-y^*(k))^2]=𝔼[w^2(1)].



	By Theorem <ref>, we have θ̂(k)→θ a.s. Then based on (<ref>), we obtain




	
    ϕ^⊺(k)θ-y^*(k)=   ϕ^⊺(k)(θ-θ̂(k-1))+ϕ^⊺(k)θ̂(k-1)-y^*(k)→ 0, a.s.

	According to Assumptions <ref>, <ref> and Proposition <ref>, we have
	
    ϕ^⊺(k)θ-y^*(k)≤M̅M+y̅^*.

	Also, by Lebesgue Dominated Convergence Theorem, we can obtain
	
    lim_k→∞𝔼[(ϕ^⊺(k)θ-y^*(k))^2]=0.

	Therefore, by (<ref>) and Assumption <ref>, we have
	
    lim_k→∞𝔼[(y(k)-y^*(k))^2]=   lim_k→∞𝔼[(ϕ^⊺(k)θ-y^*(k)+w(k))^2]
    
    			=   lim_k→∞𝔼[(ϕ^⊺(k)θ-y^*(k))^2]+𝔼[w^2(k)]
    
    			=   𝔼[w^2(1)].

	This completes the proof.




§ SIMULATIONS

In this section, we give a simulation to illustrate the main results in this paper. 

Example. Consider the following stochastic regression model:

    y(k)=u(k)θ(1)+u(k-1)θ(2)+w(k),k=1,2,...

where ϕ(k)≜[u(k),u(k-1)]^⊺∈ℝ^2 and θ≜[θ(1),θ(2)]^⊺∈ℝ^2 denote the system input and the unknown parameter, respectively, u(z)=0 for z≤0, {w(k), k≥ 1} be an i.i.d. sequence of standard normal random variables. The multi-threshold of sensor and the corresponding weight of the quantized outputs is [-∞,-2,0,2,∞] and [80,50,-50,-80]. The reference signal {y^*(k)}_k≥ 1 is generated y^*(2k)=2+e(k) and y^*(2k-1)=1, where {e(k)}_k≥ 1 is randomly selected in the interval (0,0.1). Set the true parameter θ=[4,1]^⊺ be unknown but known as in Ω≜{[x,y]:  |x|≤ 6, |y|≤ 2} and N=3.

Let θ̂(0)=[5,0]^⊺. Then we use the algorithm (<ref>)-(<ref>) to estimate the unknown θ and track the reference signal {y^*(k)}. Figure 1 and Figure 2 show that estimation and tracking effects of the algorithm, respectively. From these, we learn that the estimation algorithm is convergent (see Figure 1(a)), the mean square
convergence speed can reach O(1/k) (see Figure 1(b)), and the adaptive tracking control law can also achieve asymptotically optimal tracking for the non-periodic reference signal (see Figure 2). These results are consistent with Theorems <ref> and <ref>.





§ CONCLUSION

The motivation of this paper is to solve the adaptive tracking control problem under quantized observations for non-periodic reference signals. This paper presents an adaptive tracking control scheme that incorporates an online stochastic approximation-type estimation algorithm, and importantly this result is free from the reliance on periodic reference signals in the existing literature. The methodological innovation of this paper lies in overcoming the difficulty of ensuring that the adaptive tracking control designed by the certainty equivalent principle satisfies uniformly bounded and persistent excitation conditions by introducing two backward-shifted polynomials with time-varying parameters and a special projection structure. Finally, we obtain some results including the convergence speed of the estimation algorithm and asymptotically optimality of adaptive tracking control law for the non-periodic reference signals.

In the future, our work will focused on the following two areas: i) how to ensure that the designed adaptive tracking control satisfies the excitation condition without relying on the projection technique in this paper to achieve asymptotically optimal tracking of the non-periodic reference signal; ii) for more general high-order stochastic regression system containing quantized observations, how to extend the adaptive control scheme proposed in this paper to solve the tracking control problem for non-periodic reference signals.




















00




Calamai-More P. H. Calamai and J. J. Moré, “Projected gradient methods for linearly constrained problems,” Mathematical Programming, vol. 39, 93-116, 1987.


Chen-Guo-1 H. F. Chen and L. Guo, “Asymptotically optimal adaptive control with consistent parameter estimates,” SIAM Journal on Control and Optimization, vol 25, pp. 558-575, 1987.

Guo-Chen L. Guo and H. F. Chen, “The Åström-Wittenmark self-tuning regulator revisited and ELS-based adaptive trackers,” IEEE Transactions on Automatic Control, vol 36, pp. 802-812, 1991.

Guo L. Guo, “Further results on least squares based adaptive minimum variance control,” SIAM Journal on Control and Optimization, vol. 32, pp. 187-212, 1994.

Guo-1 L. Guo, “Convergence and logarithm laws of self-tuning regulators,” Automatica, vol 31, pp. 435-450, 1995.

Guo-2 L. Guo, “Estimation, control, and games of dynamical systems with uncertainty”(in Chinese), Science China Information Sciences, 50, pp. 1327-1344, 2020.

Guo-Zhao-1 J. Guo and Y. L. Zhao, “Recursive projection algorithm on FIR system identification with binary-valued observations,” Automatica, vol. 49, pp. 3396-3401, 2013.





Guo-Zhang-ZhaoJ. Guo, J. F, Zhang and Y. L. Zhao, “Adaptive tracking of a class of first-order systems with binary-valued observations and fixed thresholds,” Journal of Systems Science and Complexity, vol 25, pp. 1041-1051, 2012.

KongC. Kong, Y. Wang and Y. L. Zhao, “Adaptive tracking control under quantized observations and observation uncertainty with unbounded variance,” Asian Journal of Control, pp. 1-14, 2023. 

Li G. Li, “Adaptive quantized control for a class of high-order nonlinear systems,” International Journal of Robust and Nonlinear Control, vol 33, 2023.

Peter-Lafortune C. Peter and S. Lafortune,	“Adaptive control with recursive identification for stochastic linear systems,” IEEE Transactions on Automatic Control, vol 29, pp. 312-321, 1984.

Shi W. Shi, M. Hou, G. Duan and M. Hao, “Adaptive dynamic surface asymptotic tracking control of uncertain strict-feedback systems with guaranteed transient performance and accurate parameter estimation,” International Journal of Robust and Nonlinear Control, vol 32, 2022.

Wang-Lai F. Wang and G. Y. Lai, “Fixed-time control design for nonlinear uncertain systems via adaptive method,” Systems and Control Letters, vol 140, 2020.

Wang-Hu-ZhaoT. Wang, M. Hu and Y. L. Zhao, “Adaptive tracking control of FIR systems under binary-valued observations and recursive projection identification,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, pp. 5289-5299, 2021.

Wang-Kim-Sun L. Y. Wang, Y. W. Kim and J. Sun, “Prediction of oxygen storage capacity and stored NOx by HEGO sensors for improved LNT control strategies,” in Proc. ASME Int. Mech. Eng. Congr. Expo., pp. 17-22, 2002.

Wen-Tao L. Y. Wen, G. Tao and G. Song, “Higher-order tracking properties of nonlinear adaptive control systems,” Systems and Control Letters, vol 145, 2020.



Wang-Zhang-YinL. Y. Wang, J. F. Zhang and G. Yin, “System identification using binary sensors,” IEEE Transactions on Automatic Control, vol. 48, pp. 1892-1907, 2003.


Wang-Hu-Zhao-DX. Wang, M. Hu, Y. L. Zhao and B. Djehiche, “Credit scoring based on the set-valued identification method”, Journal of Systems Science and Complexity, vol. 33, pp. 1297-1309, 2020.


Zhao-Guo-ZhangY. L. Zhao, J. Guo and J. F. Zhang, “Adaptive tracking control of linear systems with binary-valued observations and periodic target,” IEEE Transactions on Automatic Control, vol. 58, pp. 1293-1298, 2013.

Zhang-Guo Y. Zhang and L. Guo, “Stochastic adaptive switching control based on multiple models,” Journal of Systems Science and Complexity, vol. 15, pp. 18–34, 2002.

Zhao-Wang-Bi-1Y. L. Zhao, T. Wang and W. Bi, “Consensus protocol for multi-agent systems with undirected topologies and binary-valued communications”, IEEE Transactions on Automatic Control, vol. 64, pp. 206-221, 2019.

Zhang-Tao Y. Zhang, G. Tao, M. Chen, W. Chen and Z. Zhang, “An implicit function-based adaptive control scheme for non-canonical-form discrete-time neural-network systems,” IEEE Transactions on Cybernetics, vol 51, pp. 5728-5739, 2021.

Zhang-Zhang Y. Zhang, J. F. Zhang and X. Liu, “Implicit function based adaptive control of non-canonical form discrete-time nonlinear systems,” Automatica, vol 129, 2021.

ChenH. F. Chen, Stochastic approximation and its application, Dordrecht, The Netherlands: Kluwer, 2002.

Chen-Guo H. F. Chen and L. Guo, Identification and Stochastic Adaptive Control, Boston, MA, USA: Birkäuser, 1991.

Wang-Yin-Zhang-Zhao L. Y. Wang, G. Yin, J. F. Zhang and Y. L. Zhao. System identification with quantized observation. Birkhauser. 2010, Boston.


