Abstract
In this paper, we analyze economic model predictive control schemes without terminal constraints, where the optimal operating regime is not steady-state operation, but periodic behavior. We first show by means of two counterexamples, that a classical 1-step receding horizon control scheme does not necessarily result in an optimal closed-loop performance. Instead, a multi-step MPC scheme may be needed in order to establish near optimal performance of the closedloop system. This behavior is analyzed in detail, and we derive checkable dissipativity-like conditions in order to obtain closedloop performance guarantees.
I. INTRODUCTION
In recent years, the study of economic model predictive control (MPC) schemes has received a significant amount of attention. In contrast to standard stabilizing MPC, the control objective is the minimization of some general performance criterion, which needs not be related to any specific steadystate to be stabilized. In the literature, closed-loop properties such as performance estimates and convergence issues have been studied for various economic MPC schemes, e.g., in [1, 2, 9, 12] using suitable additional (terminal) constraints, or in [6] without terminal constraints (see also the recent survey article [4] ).
A distinctive feature of economic MPC is the fact that the closed-loop trajectories are not necessarily convergent, but can exhibit some more complex, e.g., periodic, behavior. In particular, the optimal operating regime for a given system depends on its dynamics, the considered performance criterion and the constraints which need to be satisfied. The case where steady-state operation is optimal is by now fairly well understood, and various closed-loop guarantees have been established in this case. For example, a certain dissipativity property is both sufficient [2] and (under a mild controllability condition) necessary [13] for a system to be optimally xdavis@example.com. The same dissipativity condition (strengthened to strict dissipativity) was used in [1, 2] to prove asymptotic stability of the optimal steady-state for the resulting closed-loop system with the help of suitable terminal constraints. Similar (practical) stability results were established in [6, 7] without such terminal constraints.
On the other hand, the picture is still much less complete in case that some non-stationary behavior is the optimal operating regime. In [2] , it was shown that when using some periodic orbit as (periodic) terminal constraint within the economic MPC problem formulation, then the resulting closed-loop system will have an asymptotic average performance which is at least as good as the average cost of the periodic orbit. Convergence to the optimal periodic orbit was established in [10, 15] using similar terminal constraints, and in [11] for linear systems and convex cost functions using less restrictive generalized periodic terminal constraints. Furthermore, dissipativity conditions which are suited as sufficient conditions such that the optimal operating regime of a system is some periodic orbit were recently proposed in [8] .
In this paper, we consider economic MPC without terminal constraints for the case where periodic operation is optimal. Using no terminal constraints is in particular desirable in this case as the optimal periodic orbit then needs not be known a priori (i.e., for implementing the economic MPC scheme). Furthermore, the online computational burden might be lower and a larger feasible region is in general obtained. We first show by means of two counterexamples (see Section III), that the classical receding horizon control scheme, consisting of applying the first step of the optimal predicted input sequence to the system at each time, does not necessarily result in an optimal closed-loop performance. We then prove in Section IV that this undesirable behavior can be resolved by possibly using a multi-step MPC scheme instead. In particular, we show that the resulting closed-loop system has an asymptotic average performance which is equal to the average cost of the optimal periodic orbit (up to an error term which vanishes as the prediction horizon increases). This recovers the results of [2] , where periodic terminal constraints were used as discussed above. Furthermore, in Section V we derive checkable sufficient conditions in order to apply the results of Section IV. Our analysis builds on the one in [6] , where closed-loop performance guarantees and convergence results for economic MPC without terminal constraints were established for the case where the optimal operating regime is steady-state operation. However, while some of the employed concepts and ideas are similar to those in [6] , various properties of predicted and closed-loop sequences are different in the periodic case considered in this paper, and hence also different analysis methods are required. Finally, we note that the proofs of several results are omitted in this conference paper due to space restrictions (in particular, those of Theorems 8, 14, and 16). These proofs as well as further results and discussions can be found in [14] .
II. PRELIMINARIES AND SETUP
Let I [a,b] denote the set of integers in the interval [a, b] ⊆ R, and I ≥a the set of integers greater than or equal to a. For a ∈ R, ⌊a⌋ is defined as the largest integer smaller than or equal to a. The cardinality of a set A is denoted by #A. The distance of a point x ∈ R n to a set A ⊆ R n is defined as |x| A := inf a∈A |x − a|. For a set A ⊆ R n and ε > 0, denote by B ε (A) := {x ∈ R n : |x| A ≤ ε}. By L we denote the set of functions ϕ : R ≥0 → R ≥0 which are continuous, nonincreasing and satisfy lim k→∞ ϕ(k) = 0. Furthermore, by KL we denote the set of functions γ :
Note that the definition of a KL-function requires weaker properties than those for classical KL-functions, i.e., each KL-function is also a KLfunction (but the converse does not hold).
We consider nonlinear discrete-time systems of the form
with k ∈ I ≥0 and f : R n × R m → R n . System (1) is subject to pointwise-in-time state and input constraints
the corresponding solution of system (1) with initial condition x u (0, x) = x. For a given x ∈ X, the set of all feasible control sequences of length N is denoted by U N (x), where a feasible control sequence is such that u(k) ∈ U for all k ∈ I [0,N −1] and x u (k, x) ∈ X for all k ∈ I [0,N ] . Similarly, the set of all feasible control sequences of infinite length is denoted by U ∞ (x). In the following, we assume for simplicity that U ∞ (x) = ∅ for all x ∈ X.
Remark 1: For ease of presentation, we use decoupled state and input constraint sets X and U in the statement of our results. Nevertheless, all results in this paper are also valid for possibly coupled state and input constraints, i.e., (x(k), u(k)) ∈ Z for all k ∈ I ≥0 and some Z ⊆ R n × R m , which will also be used in the examples.
System (1) is equipped with a stage cost function ℓ : X × U → R, which is assumed to be bounded from below on X× U, i.e., ℓ min := inf x∈X,u∈U ℓ(x, u) is finite. Note that this is, e.g., the case if X×U is compact and ℓ is continuous. Without loss of generality, in the following we assume that ℓ min ≥ 0. We then define the following finite horizon averaged cost functional
and the corresponding optimal value function
In the following, we assume that for each x ∈ X, a control sequence u * N,x ∈ U N (x) exists such that the infimum in (3) is attained, i.e., u * N,x satisfies V N (x) = J N (x, u * N,x ). A standard MPC scheme without additional terminal cost and terminal constraints then consists of minimizing, at each time instant k ∈ I ≥0 with current system state x = x(k), the cost functional 1 (2) with respect to u ∈ U N (x) and applying the first part of the resulting optimal input sequence u * N,x to the system. This means that the resulting receding horizon control input to system (1) is given by
, where x uMP C (·, x) denotes the corresponding closed-loop state sequence. The finite and infinite horizon averaged cost functionals along this closed-loop state sequence are denoted by
In [6] , it was shown that if system (1) is optimally operated at some steady-state (x * , u * ) with cost ℓ 0 := ℓ(x * , u * ), then under suitable conditions the asymptotic average performance of the closed-loop system, J cl ∞ , equals ℓ 0 (up to an error term which vanishes as N → ∞). In this paper, we consider the more general case where system (1) is optimally operated at some periodic orbit with period P ∈ I ≥1 . To this end, consider the following definitions.
Definition 2: A set of state/input pairs
, and
In the following, denote by Π X the projection of Π on X, i.e., Π X := {x p 0 , . . . , x p P −1 }. Definition 3: System (1) is optimally operated at a periodic orbit Π if for each x ∈ X and each u ∈ U ∞ (x) the following inequality holds:
Definition 3 means that each feasible solution will result in an asymptotic average performance which is as good as or worse than the average performance of the periodic orbit Π. Furthermore, for P = 1 the notion of optimal steady-state operation [2, 13] is recovered. Note that if system (1) is optimally operated at some periodic orbit
Π is necessarily an optimal periodic orbit for system (1), i.e. we have
where S P Π denotes the set of all feasible P -periodic orbits. In case that a system is optimally operated at a periodic orbit Π, the closed-loop system resulting from application of the economic MPC scheme exhibits optimal performance if J cl ∞ (x, u MP C ) = (1/P )
As discussed in the Introduction, in [2] it was shown that this can be achieved in case that Π X is used as a periodic terminal constraint. When using no terminal constraints, this equality is in general not achieved, as we show in the following by means of some counterexamples. Nevertheless, optimal performance can still be guaranteed also without terminal constraints in case a multi-step MPC scheme is used, as will be shown in Sections IV and V.
III. MOTIVATING EXAMPLES
Example 4: Consider the one-dimensional system x(k + 1) = u(k) with state and input constraint set Z = {(−1, −1), (−1, 0), (0, 1), (1, 0)} consisting of four elements only and cost ℓ(x, u) defined as
for some constant ε > 0. The system is optimally operated at the two-periodic orbit given by Π = {(0, 1), (1, 0)}, and with average cost ℓ 0 := (1/2) (·, x 0 ) stays on Π X . But this means that the closed-loop system stays at x = −1 for all times, i.e., x uMP C (k, x 0 ) = −1 for all k ∈ I ≥0 , and hence
In Example 4, the non-optimal behavior of the closed-loop system for all odd prediction horizons was due to the fact that the cost on the optimal periodic orbit was not constant. The following example shows that the same behavior can occur even if the cost along the optimal periodic orbit is constant. Namely, consider again the system x(k + 1) = u(k) with state and input constraint set Z = {(−1, −1), (−1, 0), (0, 1), (1, 0), (1, 2), (2, 1)} consisting of six elements only and cost ℓ(x, u) defined as
for some constant 0 < ε < 3. The system is again optimally operated at the two-periodic orbit given by Π = {(0, 1), (1, 0)} with average cost ℓ 0 := (1/2) This means that the closed-loop system stays at x = −1 for all times, i.e., x uMP C (k, x 0 ) = −1 for all k ∈ I ≥0 , and hence J cl ∞ (−1, u MP C ) = 1 > 1 − ε = ℓ 0 . On the other hand, for any odd prediction horizon N ∈ I ≥2 similar considerations as above show that the closed-loop system converges to Π X , and hence also J
The above examples show that the "phase" on the periodic orbit is decisive, i.e., what is the optimal time to converge to the periodic orbit as well as when to leave it again. This results in the fact that one cannot guarantee that for all sufficiently large prediction horizons N , the closed-loop asymptotic average performance satisfies J cl ∞ (x, u MP C ) = (1/P )
) (plus some error term which vanishes as N → ∞), as could be established in [6] for the case of optimal steady-state operation, i.e., P = 1. On the other hand, one observes in the above examples that if the MPC scheme is modified in such a way that not only the first value of the optimal control sequence is applied to the system, but the first two values, then in both examples the closed-loop system converges to the optimal periodic orbit and hence J cl ∞ (x, u MP C ) = (1/P )
, for all prediction horizons N ∈ I ≥2 . In the following, this will be examined more closely.
IV. CLOSED-LOOP PERFORMANCE GUARANTEES
As mentioned above, in the following we consider a multistep MPC scheme where for some P ∈ I ≥1 , an optimal input sequence u * N,x is only calculated every P time instants, and then the first P elements of this sequence are applied to system (1). This means that the control input to system (1) at time k is given by
where x ′ = x uMPC (P ⌊k/P ⌋, x) and [k] := k mod P .
Remark 6:
The subsequent results are still correct if instead of the P -step MPC scheme as defined above, the following variant is used. Namely, an optimal input sequence is computed at each time and only the first element is applied to the system as in standard MPC, but the prediction horizon is periodically time-varying, i.e., N in (2) is replaced by N − [k]. By the dynamic programming principle, the closedloop sequences resulting from application of these two schemes are the same. However, the second will in general exhibit better robustness properties in case of uncertainties and disturbances, since feedback is present at each time instant and not only every P time instants.
The first result in this section for the multi-step MPC scheme as defined above is a generalization of Proposition 4.1 in [6] .
Proposition 7: Assume there exist N , P ∈ I ≥1 and δ 1 , δ 2 ∈ L such that for each x ∈ X and each N ≥ N there exists a control sequence u N,x ∈ U N +P (x) and time instants k 
(ii) There exists ℓ 0 ∈ R such that for all x ∈ X the following inequality is satisfied:
Then the inequalities
and
hold for all x ∈ X, all N ≥ N + P and all K ∈ I ≥0 . Proof: Fix x ∈ X and N ≥ N + P . Using the abbreviation x(k) = x uMP C (k, x), from the dynamic programming principle and the definition of the multi-step MPC control input in (5), we obtain that for all i ∈ I ≥0
Summing up for i = 0, . . . , K − 1 then yields
Now consider the summands in (8) . Condition (i) of the proposition with N −P in place of N and
Combining the above and defining I := {k 1 N −P,x(iP ) , . . . , k P N −P,x(iP ) }, from condition (ii) of the proposition and the definitions of
Recalling that x(0) = x and inserting (9) into (8) 
Moreover, using (9) for i = K and dividing by KP yields (6) . Finally, (7) follows from (6) by letting K → ∞ due to the fact that V N (x(KP )) ≥ ℓ min ≥ 0.
In the following, we construct control sequences u N,x such that Proposition 7 can be applied with ℓ 0 = (1/P )
for some P -periodic orbit Π. Then, inequality (7) yields the desired property that the asymptotic average performance of the closed-loop system resulting from application of the P -step MPC scheme is less than or equal to the average performance of the periodic orbit Π (up to an error term which vanishes as N → ∞). As discussed above, this approximately recovers asymptotic average performance results obtained in MPC schemes with (periodic) terminal constraints [2] .
Theorem 8: Assume that there exist constants ℓ 0 ≥ 0, δ > 0, and P ∈ I ≥1 and a set Y ⊆ X such that the following properties hold.
(a) There exists γ ℓ ∈ K ∞ such that for all δ ∈ (0,δ] and all x ∈ B δ (Y) ∩ X there exists a control sequence u x ∈ U P (x) such that the inequality (1/P )
such that for all x ∈ X and all N ∈ I ≥N1 , each optimal trajectory
Then the conditions of Proposition 7 are satisfied.
Theorem 8 uses similar conditions as Theorem 4.2 in [6] , which were shown to hold in case of optimal steady-state operation. However, there are some crucial differences. Namely, [6, Theorem 4.2] requires that |V N (x) − V N (y)| ≤ γ V (δ)/N has to hold for all y ∈ Y and all x ∈ B δ (Y) with γ V ∈ K ∞ , which in particular implies that V N (x) = V N (y) for all x, y ∈ Y, i.e., the optimal value function is constant on Y. In case that Y = Π X for some periodic orbit Π, this can in general not be satisfied, as is the case in our motivating examples in Section III. In Theorem 8, condition (b) instead only requires that
where u x is the control sequence from condition (a). Furthermore, γ V may depend on N , and in particular for fixed N , |V N (x)−V N (x ux (P, x))| needs not go to zero as δ → 0, but we only require that γ V (δ, N ) → 0 if both N → ∞ and δ → 0. These relaxations are crucial such that Theorem 8 can be applied with Y = Π X for some periodic orbit Π, as shown in the following.
V. SUFFICIENT CONDITIONS BASED ON DISSIPATIVITY AND CONTROLLABILITY
It is easy to verify that the two motivating examples satisfy the conditions of Theorem 8 with Y = Π X , which explains the fact that a 2-step MPC scheme results in optimal closedloop performance, as observed in Section III. In general, however, the conditions of Theorem 8 might be difficult to check since they involve properties of optimal trajectories and the optimal value function. The goal of this section is to provide sufficient conditions for conditions (a)-(c) of Theorem 8 for the case where Y = Π X for some periodic orbit Π of system (1) . First, we briefly discuss that condition (a) follows in a straightforward way from continuity of f and ℓ. Then, we show that a certain dissipativity-like condition results in a turnpike behavior of the system with respect to the optimal periodic orbit, from which together with suitable controllability assumptions condition (c) follows (see Section V-A). Finally, we discuss in Section V-B how condition (b) can be established under the same dissipativity and controllability assumptions.
Definition 9: System (1) is P -periodic strictly dissipative with respect to a P -periodic orbit Π if there exist storage functions λ 0 , . . . , λ P −1 : X → R ≥0 and a function α ℓ ∈ K ∞ such that
for all x ∈ X, all u ∈ U 1 (x), and all k ∈ I [0,P −1] , with λ P ≡ λ 0 .
Assumption 10 (Strict dissipativity): System (1) is Pperiodic strictly dissipative with respect to some P -periodic orbit Π, and the corresponding storage functions λ k , k ∈ I [0,P −1] , are bounded on X.
As was discussed in [8] , Assumption 10 is a sufficient condition for system (1) to be optimally operated at the periodic orbit Π.
Assumption 11 (Local controllability on B κ (Π)): There exists κ > 0, M ′ ∈ I ≥0 and ρ ∈ K ∞ such that for all z ∈ Π X and all x, y ∈ B κ (z) ∩ X there exists a control sequence u ∈ U
Assumption 12 (Finite time controllability to B κ (Π)): For κ > 0 from Assumption 11 there exists M ′′ ∈ I ≥0 such that for each x ∈ X there exists k
Before turning our attention to conditions (b) and (c) of Theorem 8, we briefly discuss how for the case that Y = Π X for some P -periodic orbit Π ⊆ int(X × U), condition (a) with ℓ 0 = (1/P )
follows from continuity of f and ℓ. In this case, for each x ∈ B δ (Y) for some δ ∈ (0,δ], by definition of Y it holds that x ∈ B δ (x p j ) for some j ∈ I [0,P −1] . Then, if f and ℓ are continuous, the control sequence u x ∈ U P in condition (a) can be chosen as
and the function γ ℓ can be computed as follows. As f and ℓ are continuous, for each compact set W ⊆ X × U there exist
Choosing W large enough and δ > 0 small enough such that
for all k ∈ I [1,P ] and condition (a) of Theorem 8 is satisfied with γ ℓ (δ) = (1/P )
A. Turnpike behavior with respect to periodic orbits
We now turn our attention to condition (c) of Theorem 8, which requires that each optimal solution is close to the set Y for at least one time instant in the interval [0, N − N 0 ]. To this end, we first state the following theorem which establishes a turnpike property [3] for system (1) with respect to a periodic orbit Π. Turnpike properties with respect to an optimal steady-state have recently been studied in the context of economic MPC both in discrete-time [6] and continuoustime [5] . The following result can be seen as a generalization to the case of time-varying periodic turnpikes.
Theorem 13: Suppose that Assumption 10 is satisfied. Then there exists C > 0 such that for each x ∈ X, each N ∈ 
Now assume for contradiction that Q ε < N −(ν +C)/α ℓ (ε).
Then there exists a set N ⊆ I [0,N −1] of N − Q ε > (ν + C)/α ℓ (ε) time instants such that |(x u (k, x), u(k))| Π > ε for all k ∈ N . By the assumption of strict dissipativity, this implies thatJ N (x, u) ≥ (N − Q ε )α ℓ (ε)/N > (ν + C)/N , which contradicts (12) and hence proves the theorem. Theorem 13 gives a lower bound Q ε for the number of time instants where the considered trajectory is "close" to the periodic orbit Π. This turnpike result can now be used together with the controllability conditions specified by Assumptions 11 and 12 to conclude condition (c) of Theorem 8, as shown in the following.
Theorem 14: Suppose that Assumptions 10-12 hold and ℓ is bounded on X × U. Then condition (c) of Theorem 8 holds for Y = Π X .
The proof proceeds by showing that controllability together with boundedness of ℓ imply existence of a constant ν > 0 such that V N (x) ≤ (1/P )

{'paper_id': '2520458', 'title': 'Economic model predictive control without terminal constraints: Optimal periodic operation', 's2_url': 'https://api.semanticscholar.org/CorpusID:2520458', 'source': 'S2ORC'}