{'fraction_non_alphanumeric': 0.0618271386267401, 'fraction_numerical': 0.015442946890653973, 'mean_word_length': 3.69516632016632, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 53, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Distributed Continuous-Time and Discrete-Time Optimization With Nonuniform Unbounded Convex Constraint Sets and Nonuniform Stepsizes

This paper is devoted to distributed continuous-time and discrete-time optimization problems with nonuniform convex constraint sets and nonuniform stepsizes for general differentiable convex objective functions. The communication graphs are not required to be strongly connected at any time, the gradients of the local objective functions are not required to be bounded when their independent variables tend to infinity, and the constraint sets are not required to be bounded. For continuous-time multi-agent systems, a distributed continuous algorithm is first introduced where the stepsizes and the convex constraint sets are both nonuniform. It is shown that all agents reach a consensus while minimizing the team objective function even when the constraint sets are unbounded. After that, the obtained results are extended to discrete-time multi-agent systems and then the case where each agent remains in a corresponding convex constraint set is studied. To ensure all agents to remain in a bounded region, a switching mechanism is introduced in the algorithms. It is shown that the distributed optimization problems can be solved, even though the discretization of the algorithms might deviate the convergence of the agents from the minimum of the objective functions. Finally, numerical examples are included to show the obtained theoretical results.


I. INTRODUCTION
As an important research direction of control theory, distributed optimization problems for multi-agent systems have attracted more and more attention from the control community [1]- [31]. The goal of a distributed optimization problem for a multi-agent system is to minimize a desired team objective function cooperatively in a distributed way where each agent can only have access to partial information of the team objective function. During the past few years, several results have been obtained for distributed optimization problems. For example, article [1] introduced a discrete-time projection algorithm for multi-agent systems with state constraints and proved that the optimization problems can be solved when the communication topologies are jointly strongly connected and balanced. Articles [4] and [5] studied a continuous-time version of the work in [1] with convex constraint sets. Article [6] gave a distributed continuous-time dynamic algorithm for distributed optimization, and subsequently, on this basis, articles [7], [8] studied the distributed optimization problem for general strongly connected balanced directed graphs and gave the estimate of the convergence rate of the algorithm. Other works about distributed optimization problems could be found in articles [2], [3], [9]-[31] and the references therein, where new algorithms, e.g., distributed Newton, approximate dual subgradient and zero-gradient-sum algorithms, were given or more complicated cases, e.g., second-order dynamics, timevarying or nonconvex functions, fixed or asynchronous stepsizes and noise, were considered.
Though many excellent results have been obtained for the distributed optimization problem, many issues need be further studied, e.g., general convex functions, nonuniform convex constraints and nonuniform stepsizes. For the issues of general convex functions and nonuniform convex constraints, most of the existing results require the gradients or subgradients of the convex functions to be bounded and the convex constraints to be identical and little attention has been paid to general convex functions and nonuniform convex constraints, in particular for multi-agent systems with general directed balanced graphs and unbounded gradients. For example, articles [7], [8] studied general convex functions but assumed them to be globally Lipschitz and the graphs are assumed to be strongly connected and balanced. Article [5] studied coercive convex functions with unbounded subgradients but the results are limited to the continuous-time multi-agent systems and the convex constraints sets are assumed to be identical for all agents. Article [1] studied nonuniform convex constraints but the communication graph is complete and all the edge weights are assumed to be equal. Founded on [1], articles [24], [25] gave some results on nonuniform convex constraints but the communication graph is constant and connected, and the objective functions are assumed to be strongly convex or some intermediate variables need be transmitted besides the agent states. Article [29] studied a distributed optimization problem with nonuniform convex constraints and gave conditions to guarantee the optimal convergence of the team objective function, but the subgradients and the convex constraint sets are both bounded. Note that [1], [5], [24], [25], [29] all adopt a uniform stepsize. For the issue of nonuniform stepsizes, currently, there are few works concerned about this issue. Articles [10], [11] studied the distributed optimization problem with nonuniform stepsizes in a stochastic setting, where the communication graphs are required to be undirected and connected. Article [30] introduced a kind of nonuniform stepsizes but the discontinuous algorithms were employed to realize the consensus of all agents and the convex constraint sets are assumed to be identical. Article [31] also studied the distributed optimization problem with nonuniform stepsizes but some intermediate variables need be transmitted besides the agent states in order to track the average of the gradients.   In this paper, we are interested in studying distributed continuous-time and discrete-time optimization problems with nonuniform convex constraint sets and nonuniform stepsizes for general differentiable convex objective functions. The communication graphs might not be strongly connected at any time and it is only required that the union of the communication graphs among the time intervals of a certain length be strongly connected. The gradients of the local objective functions considered might not be bounded when their independent variables tend to infinity. First, a distributed continuous-time algorithm is introduced where the stepsizes and the convex constraint sets are both nonuniform. Nonuniform stepsizes mean that the weights of the gradients of the local objective functions in the control input of the agents are nonuniform. That is, the optimal convergence rates of the local objective functions are different, which has great possibility to result in the destruction of the optimal convergence of the team objective function. The existing works (e.g., [1]) usually assumed the stepsizes are uniform and took no consideration of nonuniform stepsizes, and hence their approaches are hard to be applied for the case of nonuniform stepsizes. Our approach is to introduce a kind of stepsizes such that the stepsizes of each agent are constructed only based on its own states and the differences between the stepsizes of all agents vanish to zero as time evolves. Though the stepsizes of all agents tend to the same as time evolves, their differences still heavily affect the consensus stability and optimal convergence of the system, especially when the communication graphs are not strongly connected. Moreover, due to the existence of nonuniform convex constraint sets, we need take into account the nonlinearity of the consensus and optimal convergence caused by nonuniform convex constraint sets, which renders the analysis of this case to be very complicated. In particular when the nonuniform constraint sets are unbounded, the gradients of the local objective functions might tend to be unbounded when their independent variables tend to infinity, which makes the existing approaches invalid, e.g., [1], [29], where the nonuniform constraint sets and the subgradients were both bounded. To solve the optimization problem, we perform the analysis in three steps. The first step is to make full use of the convexity of the objective functions and show that our algorithm ensures that all agents remain in a bounded region for all the time. The second step is to analyze the agent dynamics at some key times and show the consensus convergence of all agents. The third step is to estimate the consensus errors and use the estimation of the distance from the agents to the convex constraint sets and the convexity of the objective functions to show the optimal convergence of the optimization problem. After that, we extend the obtained results to discrete-time multiagent systems, and then study the case where each agent remains in its corresponding convex constraint set. Due to the discretization of the system dynamics, the agents might deviate from the minimum of the objective functions. Such a problem also exists in the centralized optimization system. To deal with it, a switching mechanism is introduced in the algorithms based on each agent's own information under which all agents remain in a bounded region even when the convex constraint sets are unbounded. It is shown that the distributed optimization problems with nonuniform convex constraint sets and nonuniform stepsizes can be solved for the discrete-time multi-agent systems.
This paper takes nonuniform unbounded convex constraint sets, nonuniform stepsizes, general differentiable convex objective functions, general switching graphs and the discretization of the algorithms into account simultaneously for the distributed optimization problems. The nonlinearities caused by these factors are different and the coexistence of these nonlinearities would further result in more complicated nonlinearities. Existing works only addressed a fraction of these factors due to the limitations of the algorithms and the analytical approaches. For example, the algorithms in [7], [8] cannot be directly applied to the case of convex constraint sets due to the adoption of the integrator operator. Notation: R m denotes the set of all m dimensional real column vectors; R m×n denotes the set of all m × n real matrices; I denotes the index set {1, . . . , n}; 1 represents a column vector of all ones with a compatible dimension; s i denotes the ith component of the vector s; A ij denotes the (i, j)th entry of the matrix A; s T and A T denote, respectively, the transpose of the vector s and the matrix A; ||s|| denotes the Euclidean norm of the vector s; ∇f (s) denotes the gradient of the function f (s) at s; diag{A 1 , · · · , A n } denotes a block diagonal matrices with its diagonal blocks equal to the matrices A i (k); the symbol / denotes the division sign; and P X (s) denotes the projection of the vector s onto the closed convex set X, i.e., P X (s) = arg min s∈X s −s .

A. Preliminaries
Let G(I, E, A) be a directed communication graph of agents, where E ⊆ I × I is the set of edges, and A = [a ij ] ∈ R n×n is the weighted adjacency matrix. An edge (i, j) ∈ E denotes that agent j can obtain information from agent i. The weighted adjacency matrix A is defined as η ≤ a ij ≤η for two constantsη > η > 0 if (j, i) ∈ E and a ij = 0 otherwise. It is assumed by default that a ii = 0, i.e., (i, i) / ∈ E. The Laplacian of the graph G, denoted by L, is defined as L ii = n j=1,j =i a ij and L ij = −a ij for all i = j. The graph G is undirected if a ij = a ji for all i, j, and it is balanced if n j=1 a ij = n j=1 a ji for all i. The set of neighbors of agent i is denoted by N i = {j ∈ I | (j, i) ∈ E}. A path is a sequence of edges of the form (i 1 , i 2 ), (i 2 , i 3 ), · · · , where i j ∈ I. The graph G is strongly connected, if there is a path from every agent to every other agent, and the graph G is connected, if it is undirected and strongly connected [33].

Lemma 1.
[33] If the graph G is strongly connected, the Laplacian L has one zero eigenvalue associated with eigenvalue vector 1 and all its rest n − 1 eigenvalues have positive real parts. Further, if the graph G is undirected and connected, all the n − 1 nonzero eigenvalues are positive.

Lemma 3.
[35] Suppose that Y = ∅ is a closed convex set in R r . The following statements hold.
(1) For any y ∈ R r , y − P Y (y) is continuous with respect to y and ∇ 1 2 y − P Y (y) 2 = y − P Y (y); (2) For any y, z ∈ R r and all ξ ∈ Y ,

B. Problem formulation
Consider a multi-agent system consisting of n agents. Each agent is regarded as a node in a directed graph G 1 , and each agent can interact with only its local neighbors in G. Our objective is to design algorithms using only local interaction and information such that all agents cooperatively find the optimal state that solves the optimization problem where f i (s) : R r → R denotes the differentiable convex local objective function of agent i, and H i denotes the closed convex constraint set of f i (s). Clearly, n i=1 f i (s) is also a 1 In the following, G(t), G(kT ), a ij (t), a ij (kT ), N i (t) and N i (kT ) will be used to denote the graph, the edge weight and the agent neighbor set at time t or k as defined in Sec. II.A. differentiable convex function. It is assumed that f i (s) and H i are known only to agent i. The problem described above is equivalent to the problem that all agents reach a consensus while minimizing the team objective function (1) In this paper, our analysis is for the general m case. When no confusion arises, the equations or formula are written in the form of m = 1 for notational simplicity.

III. DISTRIBUTED CONTINUOUS-TIME OPTIMIZATION
In this section, we discuss the distributed optimization problem for continuous-time multi-agent systems. The problem has applications in motion coordination of multi-agent systems, where multiple physical vehicles rendezvous or form a formation centered at a team optimal location. Suppose that the agents satisfy the continuous-time dynamicṡ where x i ∈ R r is the state of agent i, and u i ∈ R r is the control input of agent i.

A. Assumptions and some necessary lemmas
Let X denote the optimal set of the constrained optimization problem (1). Before the main assumptions and the necessary lemmas, we need further define the sets X i x ∇f i (x) = 0 for all i ∈ I and X x n i=1 ∇f i (x) = 0 for later usage. From Lemma 2, X i and X are convex and denote, respectively, the optimal sets of f i (x) and the team objective

Assumption 1.
[30] Each set X i , i ∈ I, is nonempty and bounded.
In Assumption 1, we only make an assumption on each f i (x) rather than the team objective function is global information for all agents and cannot be used by each agent in a distributed way.
In Assumption 2, we do not require H i to be bounded and hence H might be unbounded. (2) All X i , i ∈ I, and X are nonempty closed bounded convex sets.
Actually, Lemma 5 shows the existence and boundedness of the optimal set of the constrained optimization problem (1), X , under Assumptions 1 and 2.

Assumption 3.
[30] The length of the time interval between two contiguous switching times is no smaller than a given constant, denoted by d w .
Under Assumption 3, the switching of the graph G(t) cannot be arbitrarily fast, which prevents the system from exhibiting the Zeno behavior. Assumption 4. There exists an infinite sequence of swiching times of the graph G(t), t 0 , t 1 , t 2 , · · · , such that t 0 = 0, 0 < t j+1 − t j ≤ M and the union of all the graphs during each interval [t j , t j+1 ) is strongly connected for some constant M > d w and all nonnegative integers j.
Assumption 4 ensures that all agents can communicate with each other persistently. Suppose that the graph can be written as corresponds to a strongly connected component of the agents. From Lemma 1, each eigenvalue of is nonnegative and hence all eigenvalues of are nonnegative. Before the main results, we first present some necessary lemmas that will be used in the analysis of the main results. Specifically, Lemma 6 shows a radial growth property of the derivatives of f i (x), Lemma 7 shows the consensus convergence property of the stepsizes, Lemma 8 shows a dependency relationship between the distances from one given point to the convex sets and their intersection, and Lemma 9 shows the boundedness of the gradients in a bounded region. For clarity, the proofs of Lemmas 7-9 together with Theorems 1-4 are provided in the Appendix.

Lemma 6.
[30] Let f (s) : Ξ → R be a differentiable convex function and Y be its minimum set in Ξ, where Ξ ⊆ R r is a closed convex set. Suppose that Y ⊆ Ξ is closed and bounded. For any z = λP Y (y) Lemma 7. For the system given byq Lemma 9. Let Y be a closed bounded convex set. Then, ∇f i (x) < ̺ for all i, all x ∈ Y and some constant ̺ > 0.

B. Algorithm and convergence analysis
In this subsection, we design a continuous distributed optimization algorithms with nonuniform stepsizes. The algorithm is given bẏ for all i. The stepsize of the gradient, 1/ q i (t), is used to make the term ∇fi(xi(t)) √ qi(t) tend to zero as t → +∞. The role is to make all agents converge to the optimal set of the team objective function and the role of −[x i (t) − P Hi (x i (t))] is to make each agent converge to the convex set H i .
It should be noted that the construction of the stepsize 1/ q i (t) is only based on the ith agent's own states and it does not use the Lipschitz constant or the convexity constant as in the existing works, e.g., [7] and [24]. The stepsize of each agent 1/ q i (t) is state-dependent and can be nonuniform for all agents. The existing works often assume the stepsizes of all agents to be predesigned and consistent with each other at any time, which exerts a heavy burden on sensing and communication costs of the entire system. Remark 1. In algorithm (3), the role of the inverse tangent functions and the exponential functions is to ensureq i (t) to be upper and lower bounded. In fact, some other more general functions, e.g., saturation function, can be employed to play the same role. Moreover, the stepsizes used in algorithm (3) are in a special form, and it can also be extended to other functions. For easy readability, we do not give the general form of the functions q i (t) and the stepsizes. Let , · · · , x n (t) − P Hn (x n (t))] T . Then the system (2) with (3) can be written asẋ For convenience of discussion, under Assumption 4, let t j0 < t j1 < · · · < t jm j with t j0 = t j and t jm j = t j+1 denote all the switching times in the interval [t j , t j+1 ). In the following, we show the effectiveness of this algorithm to solve the optimization problem (1). Theorem 1. Suppose that the graph G(t) is balanced for all t and Assumptions 1, 2, 3 and 4 hold. For arbitrary initial conditions x i (0) ∈ R r , using algorithm (3) for system (2), the following statements hold.
(1) All x i (t) remain in a bounded region for all i and all t.
Theorem 2. Suppose that the graph G(t) is balanced for all t and Assumptions 1, 2, 3 and 4 hold. For arbitrary initial conditions x i (0) ∈ R r , using algorithm (3) for system (2), all agents reach a consensus, i.e., lim t→+∞ [x i (t)−x * (t)] = 0 for all i, and minimize the team objective function (1) as t → +∞.
Remark 2. In the existing works, the distributed optimization problems were considered often under the assumption that the stepsizes of the gradients are uniform and explicitly timedependent for all agents. That is, the stepsizes should be consistent with each other at any time. In algorithm (3), we do not make such an assumption and the stepsizes of the gradients only depend on the agent states. The stepsizes need not have the same value at any time and instead they are usually nonuniform, which greatly relaxes the synchronization requirement on the system.

IV. DISTRIBUTED DISCRETE-TIME OPTIMIZATION
In this section, we discuss the distributed optimization problem for discrete-time multi-agent systems. Suppose that the agents satisfy the discrete-time dynamics where x i ∈ R r is the state of agent i, u i ∈ R r is the control input of agent i, and T > 0 is the sample time. In the following, we use "(k)" instead of "(kT )" when no confusion arises.

A. Assumptions and some necessary lemmas
In Sec. II, when we define the weighted adjacency matrix A, we assume by default that a ii = 0. In this section, for discussion of discrete-time multi-agent systems, we need to redefine a ii and make an assumption about the weighted adjacency matrix A as shown in the following assumption.

Assumption 5.
[29] For all i, j ∈ I, a ii (k) ≥ η, a ij (k) ≥ η for some constant 0 < η ≤ 1 and each nonzero a ij (k), n i=1 a ij (k) = 1 and n j=1 a ji (k) = 1. Under Assumption 5, the adjacency matrix of the graph G(k) is doubly stochastic and its diagonal entries are nonzero. Assumption 5 is used to generate convex combinations of the agents' states such that the influence of each agent's state is equal in the final consensus value in the distributed optimization algorithms shown later.

Assumption 6.
[29] There exists an infinite sequence of switching times of the graph G(k), k 0 , k 1 , k 2 , · · · , such that k 0 = 0, 0 < k j+1 − k j ≤ M and the union of all the graphs during each interval [k j , k j+1 ) is strongly connected for some positive integer M and all nonnegative integers j.
Similar to Assumption 4, Assumption 6 ensures that all agents can communicate with each other persistently.

B. Distributed optimization with nonuniform stepsizes
In this subsection, we design a discrete-time distributed optimization algorithm with nonuniform stepsizes. The algorithm is given by where 0 < γ i ≤ 1 is a constant for each i.
Due to the discretization of the system dynamics, the agents might deviate from the minimum of the team objective function. Such a problem also exists in the centralized optimization system. To deal with it, a switching mechanism is introduced in (6) based on each agent' own information, under which the gradient term would not be too large to result in the divergence of the system. This will be shown in the proof of Theorem 3.
For Algorithm (6), it can be calculated simply in four steps: (a) v i (k), q i (k + 1) and ∇f i (v i (k)); (b) gr i (k) based on the switching mechanism; (c) w i (k); and (d) P Hi [w i (k)] and u i (k). Though the algorithm computation looks a bit complex due to the existence of the switching mechanism, the algorithm does not require intermediate variables to be transmitted and it is a fully distributed algorithm.
In Theorems 2 and 3, it is not required that each agent remain in its corresponding convex constraint set H i . In the following theorem, we show that the optimization problem (1) can be solved when all agents remain in their corresponding convex constraint sets. Remark 3. Since the proposed algorithms are gradient based, the convergence rate of the algorithms is not very fast. This is common for the gradient-based distributed algorithms in the existing literature. In particular, the stepsizes (gradient gains) are nonuniform, which makes the convergence rate slower than that with uniform stepsizes. However, our algorithms are able to deal with the general case of nonuniform stepsizes without intermediate variables being transmitted. In the existing nongradient-based works, some special assumptions are always made in order to ensure the optimal convergence. For example, in [9], the communication graphs are assumed to be strongly connected and the the convex objective functions are assumed to be strongly convex. In this paper, the communication graphs are only required to be jointly strongly connected, and the convex objective functions are only required to be differentiable (which can be easily extended to the nondifferentiable case by using subgradients). Future work could be directed towards improving the convergence rate of our algorithms. In particular, different dimension might yield different convergence rates. It is worth studying the effects of different dimensions on the convergence rate of the algorithms. x i1 (k) x i2 (k) x 4 (0) x 5 (0) x 6 (0) x 7 (0) The optimal point (−0.5,1)

V. SIMULATIONS
Consider a multi-agent system with 8 agents in R 2 . The communication graphs switch among the balanced subgraphs of the graph shown in Fig. 1. Each edge weight is 0.5. The sample time is T = 0.1 s for the discrete-time algorithm. The local objective functions are adopted as f 1 (  (6), all agents remain in their corresponding constraint sets. All the simulation results are consistent with our obtained theorems.

VI. CONCLUSIONS
In this paper, distributed continuous-time and discrete-time optimization problems with nonuniform stepsizes and nonuniform possibly unbounded convex constraint sets were studied for general differentiable convex objective functions. One continuous-time algorithm and one discrete-time algorithm were introduced. For these two algorithms, it has been shown that the team objective function is minimized with nonuniform stepsizes and nonuniform convex constraint sets. In particular, for the discrete-time algorithm, it has been shown that the distributed optimization problem can be solved when each agent remain in its corresponding constraint set.
Note that H ψ ∩ Y is a closed bounded set. The inequality y − P H (y) = ψ > 0 implies that the function n i=1 y − P Hi (y) on H ψ ∩ Y is lower bounded by a positive constant, denoted by γ. If this is not true, there must exist a sequence This contradicts with the definition of H ψ .
Since y(t) ∈ Y for all t, lim k→+∞ y(t k ) − P H (y(t k )) = ψ and lim t→+∞ y(t) − P Hi (y(t)) = 0 for all i, there exists a constant N > 0 for any ǫ > 0 such that n i=1 y(t k ) − P Hi (y(t k )) −γ > −ǫ and y(t)−P Hi (y(t)) < ǫ for all t k > N and t > N . Let ǫ < γ n+1 . It follows that n i=1 y(t k ) − P Hi (y(t k )) > γ − ǫ > nǫ > n i=1 y(t) − P Hi (y(t)) for all t k > N and t > N , which is a contradiction. Proof of Lemma 9: Since each function f i (x) is differentiable, ∇f i (x) exists for any x ∈ Y . If ∇f i (x) is unbounded for some x ∈ Y , there must exist a sequence {y k ∈ Y, k = 1, 2, · · · } such that lim k→+∞ y k = y 0 for some y 0 ∈ Y and lim k→+∞ ∇f i (y k ) = +∞. Let z k = y k +∇f i (y k )/ ∇f i (y k ) . It is clear that each f i (z k ) and each f i (y k ) are bounded. From the convexity of f i (y k ), it follows Proof of Theorem 1: First, we prove that using (3) for (2) all x i (t) remain in a bounded region and all ∇f i (x i (t)) are bounded for all i and all t. Note that π/4 ≤ arctan(e xi(t) ) ≤ π/2 for all t and all i. There exists a constant T 0 > 0 such that 2 √ t > q i (t) > √ t 2 for all i and all t ≥ T 0 . Under Assumption 1, from Lemma 4, X and all X i are nonempty and bounded. Note from Lemma 5 that X is nonempty and bounded under Assumptions 1 and 2. There is a closed bounded convex set Y = {y | y − P X (y) ≤ P 1 } ⊂ R r for some constant P 1 > 0 such that x i (T 0 ) ∈ Y , X ⊂ Y , X i ⊂ Y and X ⊂ Y for all i. From the continuity of f i (x) and Lemma 4, let Y be sufficiently large such that Consider the Lyapunov function candidate V 0 (t) = 1 2 x(t) − z1 2 for some z ∈ X and t ≥ T 0 . Calculatinġ V 0 (t), we havė for t ≥ T 0 . Since the graph G(t) is balanced, then 1 T L(t) = 0 and hence z1 T L(t)x(t) = 0. Regarding L(t) T +L(t) 2 as a Laplacian of an undirected graph, we have − x(t) ≤ 0 from the previous analysis below Assumption 4. Since the function 1 2 for t ≥ T 0 . From the convexity of the function f i (x i (t)), we have .
As a result, we haveV 0 (t) ≤ 0 for t ≥ T 0 if there exists an agent i 0 such that x i0 (t) / ∈ Y for some i 0 . This implies that all x i (t) remain in a bounded region for all i and all t. Thus, is bounded. Further, from Lemma 9, it follows that ∇f i (x i (t)) is bounded for all i and all t.
2 for all i and all t ≥ T 0 , we have for all i, all t ≥ T 0 and some constant ρ > 0. Moreover, since the graph G(t) is balanced, ].
Consider the Lyapunov function candidate for all t ≥ T 0 . CalculatingV 1 (t), we havė Tẋ * (t) = 0, and the last inequality has used the convexity of x i (t) − P Hi (x i (t)) 2 and the fact that To proceed with our proof, we prove that V 1 (t) ≤ δ for any constant δ > 0 when t is sufficiently large. Since all x i (t) remain in a bounded region for all i and all t, each ẋ i (t) is bounded for all i, t. There exists a constant h 0 > 0 such that and t k ≥ T 0 where t k has been defined in Assumption 4. Suppose that h 1 (t k ) = max s∈[t k ,t k+1 ] φ(s) for t k ≥ T 0 . Note that it takes at least h 1 (t k )/h 0 time for the value of φ(t) to vary from h 1 (t k ) to 0 or from 0 to h 1 (t k ). By simple Let T 0 be sufficiently large for some ǫ < d 2 Note that . Under Assumption 4, the union of all the graphs during each interval [t k , t k+1 ), denoted byĜ(t k ), is strongly connected. InĜ(t k ), there must exist a directed path between every two agents whose length is no larger than n. Thus, From Lemma 8, when lim t→+∞ x * (t) − P Hi (x * (t)) = 0 for all i, lim t→+∞ x * (t) − P H (x * (t)) = 0. Let Then there exists a constant δ 1 > 0 for any ǫ 1 > 0 such that Let ǫ be sufficiently small such that . Thus, there exists a constant T 1 > T 0 such that V (t k ) < ǫ 1 for all t k > T 1 . In view of the arbitrariness of ǫ 1 , let ǫ 1 → 0. It follows that lim k→+∞ V 1 (t k ) = 0.
Proof of Theorem 2: Note that π/4 ≤ arctan(e xi(t) ) ≤ π/2 for all t and all i. There exists a constant T 0 > 0 such that for all i and all t ≥ T 0 . From Theorem 1, all x i (t) remain in a bounded region, and lim t→+∞ [x i (t)−x * (t)] = lim t→+∞ x * (t)−P H (x * (t)) = 0 for all i. It is clear that x * (t) remains in a bounded region. Moreover, x * (t) − s is bounded for some s ∈ H and hence x * (t) − P H (x * (t)) is bounded for all t. From Lemma 5, X is a nonempty closed bounded convex set. It follows that f i (x * (t)) and f i (P X (x * (t))) are both bounded for all i and all t. Therefore, together with Lemma 7, it follows that there exists a constant T 1 > T 0 for any ǫ 0 > 0 such that for all i, j and all t ≥ T 1 .
Using the convexity of the functions f i (s), it follows thaṫ where the last two inequalities has used the fact that √ t 2 < q i (t) < 2 √ t for all i and t ≥ T 0 . In the following, we use the properties of the point P H (x * (t)) to show the optimal convergence of the system. Define l 0 = max s∈H s − P X (s) . Let Let ǫ 0 be sufficiently small such that 2ǫ 0 + 10nǫ 0 < β. It follows that for any t ≥ T 1 and any x * (t) − P X (x * (t)) > l 1 + ǫ 0 , Integrating both sides of this inequality from T 1 to t, we have Note that (12) holds for any t ≥ T 1 and any x * (t) − P X (x * (t)) > l 1 + ǫ 0 . Thus, for all t ≥ T 2 . In view of the arbitrariness of l 1 and ǫ 0 , letting l 1 → 0 and ǫ 0 → 0, we have That is, all agents reach a consensus and minimize the team objective function (1) as t → +∞. Proof of Theorem 3: Note that π/4 ≤ arctan(e xi(k) ) ≤ π/2 for all k and all i. There exists a constant T 0 > 0 such that kT 4 < q i (k) < 4kT for all k ≥ T 0 . Under Assumption 1, X and all X i are nonempty and bounded. Note from Lemma 5 that X is nonempty and bounded under Assumptions 1 and 2. There must exist a closed bounded convex set Y = {y | y − z ≤ P 2 } ⊂ R r for some z ∈ X ⊂ H and some constant P 2 > 0 such that for all i, where the first inequality has used the convexity of the function s−P Hi (s) 2 , and the second inequality has used the fact that z ∈ H ⊂ H i and z − P Hi (z) 2 = 0 and the fact that 0 < γ i < 1. When gr i (k) = 0, it follows that When for k ≥ T 0 , where the first inequality has used the convexity of the function f i (s) and the second inequality has used the fact that kT If v i (k) ∈ Y , then x i (k) − z ≤ P 2 /η and it follows from (13) and (15) that If gr i (k) = 0 and x i (k) − z > P 2 /η, then v i (k) / ∈ Y and hence It follows from (13) and (15) that x i (k + 1) − z 2 ≤ V 0 (k). If gr i (k) = 0 for some i, it follows from (13) and (14) that Summarizing the above analysis, all agents remain in a bounded region. Consequently, all x i (k) and all v i (k) are bounded. Since all f i (s) are differentiable, ∇f i (v i (k)) is bounded for all i. Then there exists a constant and V 12 (k) = n x * (k) − P H (x * (k)) 2 for k ≥ T 1 . Under Assumption 5, for k ≥ T 1 . It follows that for all k ≥ T 1 and two constants 0 <c 1 < c 1 , where under Assumption 5 and the terms such as c1 √ k and c1 k can be obtained based on the fact that the variables such as P Hi (w i (k)) , w i (k) and ∇f i (v i (k)) are all bounded and the fact that kT 4 < q i (k) < 4kT for all k ≥ T 0 . Also, for all k ≥ T 1 and some constant c 2 > 0, where the last inequality has used the convexity of 1 2 P Hi (w i (k))−w i (k) 2 , i.e., [w i (k) − P H (x * (k))] T [P Hi (w i (k)) − w i (k))] ≤ 1 2 P Hi (P H (x * (k))) − P H (x * (k)) 2 − 1 2 P Hi (w i (k)) − w i (k) 2 ≤ − 1 2 P Hi (w i (k)) − w i (k) 2 . Under Assumption 5, we have A(k)1 = 1 and hence A(k)1x * (k) = 1x * (k). It follows that Note from the form of w i (k) that k for some constant c 3 > 0 and k ≥ T 1 . Together with (16) and (17), we have that for some constant c 4 > 0 and k ≥ T 1 wherē (1 − γ i )γ i P Hi (w i (k)) − w i (k)) 2 . By repeated calculations, km for some constant c 5 > 0. Let T 1 be sufficiently large for any ǫ > 0 such that c5 √ km + c5 km < ǫ for all k ≥ T 1 and h 1 (k m ) = max k∈[km,km+1)φ (k) for k m ≥ T 1 . When h 1 (k m ) > 2ǫ for k m ≥ T 1 , Now, we need to consider the upper bound of x i (k) − x * (k) when h 1 (k m ) ≤ 2ǫ for k m ≥ T 1 . Under Assumption 5, the graph G(k) is balanced for all k. Rearranging the agent indices, it can be obtained that A(k) can be denoted as A(k) = diag{A 1 (k), · · · , A h (k)}, where each A i (k) corresponds to a strongly connected component of the agents. As a result, A(k) T A(k) = diag{A 1 (k) T A 1 (k), · · · , A h (k) T A h (k)}.
Note that n i=1 a ij (k) = n j=1 a ji (k) = 1 under Assumption 5. It follows that each row sum of A i (k) T A i (k) is 1 and hence [I − A(k) T A(k)]1 = 0. Note that A i (k) T A i (k) is symmetric and each of its off-diagonal entries is nonnegative. I − A(k) T A(k) can be regarded as the Laplacian of a certain undirected graph, denoted byḠ(k), and each I −A i (k) T A i (k) corresponds to each connected component ofḠ(k), which in turn corresponds to the strongly connected component of G(k). Under Assumption 6, the union of all G(k) during each interval [k j , k j+1 ) is strongly connected. Hence, the union of allḠ(k) during each interval [k j , k j+1 ) is connected. Note that each nonzero entry of A(k) is no smaller than η under Assumption 5 and hence each nonzero entry of A(k) T A(k) is no smaller than η 2 . Thus, By some calculations similar to (9) and (10), there exists a constant c 6 > 0 such that x i (k) − x * (k) ≤ c 6 ( √ ǫ + ǫ) and x * (k) − P Hi (x * (k)) ≤ c 6 ( √ ǫ + ǫ) for all i and all k m ≤ k < k m+1 − 1 when h 1 (k m ) ≤ 2ǫ for k m > T 1 . Let From Lemma 8, when lim t→+∞ x * (k) − P Hi (x * (k)) = 0 for all i, lim t→+∞ x * (k) − P H (x * (k)) = 0. There exists a constant δ 1 > 0 for any ǫ 1 > 0 such that V 1 (k) < ǫ 1 when φ 1 (k) < δ 1 . Let ǫ be sufficiently small such thatφ 1 (k) ≤ δ 1 and hence V 1 (k) < ǫ 1 for all k m ≤ k < k m+1 when h 1 (k m ) ≤ 2ǫ for k m > T 1 . Note that h 1 (k m ) > 2ǫ when V 1 (k) > ǫ 1 for some T 1 < k m ≤ k < k m+1 and recall that when h 1 (k m ) > 2ǫ, V 1 (k m+1 ) − V 1 (k m ) < −ǫ. Thus, there exists a constant T 1 > T 0 such that V 1 (k m ) ≤ ǫ 1 for all k m ≥ T 1 . In view of the arbitrariness of ǫ 1 , let ǫ 1 → 0. It follows that lim km→+∞ V 1 (k m ) = 0. Since k for all k > T 1 and k m+1 − k m ≤ M , it follows that lim k→+∞ V 1 (k) = lim km→+∞ V 1 (k m ) = 0. Thus, lim k→+∞ x * (k) − P H (x * (k)) = lim k→+∞ [x i (k) − x * (k)] = 0 for all i. The rest proof is very similar to that of Theorem 2 and hence omitted. Proof of Theorem 4: Since x i (0) ∈ H i and γ i = 1 for all i, from (6), x i (k) ∈ H i for all i and all k. By the same approach of the proof of Theorem 3, this theorem can be proved. However, it should be noted that x * (k) − P Hi (x * (k)) ≤ x i (k) − x * (k) ≤ c 6 ( √ ǫ + ǫ) when it has been proved that x i (k) − x * (k) ≤ c 6 ( √ ǫ + ǫ) for all i and two certain constants c 6 > 0 and ǫ > 0.