Solution Differentiability for Nonlinear Parametric Control Problems.
Perturbed nonlinear control problems with data depending  on a vector parameter are considered. Using second-order sufficient  optimality conditions, it is shown that the optimal solution and the adjoint  multipliers are differentiable functions of the parameter.  The proof  exploits the close connections between solutions of a Riccati differential  equation and shooting methods for solving the associated boundary value  problem.  Solution differentiability provides a firm theoretical basis for  numerical feedback schemes that have been developed for computing  neighbouring extremals.  The results are illustrated by an example that  admits two extremal solutions. Second-order sufficient conditions single out  one optimal solution for which a sensitivity analysis is carried out.
Introduction
This paper is concerned with parametric nonlinear control problems where all
data depend on a vector parameter p 2 IR k . In order to make the main ideas
more transparent we restrict the discussion to the following simple prototype:
Minimize J(x; u;
R
a
subject to -
The problem P (p 0 ) corresponding to a fixed parameter p 0 2 IR k is considered
as the unperturbed problem. Assume that a local minimum (optimal
problem in sensitivity analysis
is the following: find conditions for the unperturbed optimal solution x
such that the perturbed problem P (p) admits an optimal solution x(p), u(p)
near x 0 , u 0 which is a differentiable function of the parameter p near p 0 . Comparing
sensitivity approaches in optimization and optimal control it is apparent
that second-order sufficient optimality conditions (SSCs) are a crucial
assumption for this type of sensitivity result. Let us briefly review existing
papers in this regard.
In finite-dimensional nonlinear programming we have the famous second-order
sensitivity result of Fiacco and McCormick [13], [14]. This basic sensitivity
result has been extended under milder assumptions in [15] [17], [18],
[21], [25], [26], [46], [48], [49] and by other authors cited in these papers.
Semi-infinite programming problems under SSCs are treated by Rupp [47].
A direct generalization of the Fiacco and McCormick result to optimization
problems in Hilbert spaces may be found in Wiercbicki and Kurcyusz [52],
Theorem 8.6. These authors consider optimization problems with equality
constraints and finitely many inequality constraints. For Hilbert space optimization
problems including infinite-dimensional inequality constraints, Alt
[1] and Malanowski [32] have shown that the optimal solution is directionally
differentiable with respect to the parameter. These results have recently been
extended by Colonius and Kunisch [8]. The setting in Alt [1] and Malanowski
[32] allows for applications to convex control problems. A direct treatment of
convex control problems with control appearing linearly has been performed
earlier by Dontchev [11] and Malanowski [29] - [31]. Elliptic control problems
have been considered in Malanowski and Sokolowski [33].
In the framework of nonlinear control problems the second-order sensitivity
analysis goes back as far as to the ingenious papers of Breakwell and
Ho [4], Breakwell, Speyer and Bryson [5] and Kelley [19], [20]. This approach
is summarized in Bryson and Ho [6]. Similar ideas have been developed in
[9], [12], [39], [53]. However, the theory presented in [6] is rather formal and
non-rigorous. It has been indicated in Maurer [36] and Pesch [40] - [43] that
the work of these authors can be put on a firm mathematical basis using
mathematical rigorous SSCs for control problems.
We may conclude that a second-order sensitivity result for nonlinear control
problems is still lacking. It is the main purpose of this paper to provide
such a result using SSCs in [36], [38], [50], [54], [55]. Similar to finite-dimensional
programming problems, the interplay between sensitivity and
SSCs consists of the following three steps:
Step 1: Derive SSCs for the unperturbed problem and verify that these
conditions are stable with respect to small perturbations.
Step 2: Use the implicit function theorem to construct extremal solutions
satisfying the first-order necessary conditions of optimality. Verify that the
assumptions of the implicit function theorem hold if SSCs are satisfied.
Step 3: Approximate the perturbed solution x(p), u(p) by the linear ap-
proximation
dp
(p
dp
(p
The differentials dx
dp
dp
are solutions of a linear boundary value
problem (BVP). The numerical informations needed to solve this linear BVP
are generated in the process of computing the unpertubed solution x
The requirement in Step 1 is met by any of the SSCs in [36], [38], [50], [54],
[55]. SSCs depend on the existence of a finite solution of a Riccati differential
equation which is equivalent to the nonconjugate point condition. Pesch
[43] has already used the fact that the nonconjugate point condition comprises
the nonsingularity of the iteration matrix of the shooting algorithm.
The nonsingularity of this matrix is essential for constructing a family of
neighboring extremals which resolves the first part of Step 2. The implicit
function theorem has been used by many authors to establish the existence
of a family of extremals; cf. [2], [3], [22], [34], [40] - [43]. However, the
proof of optimality remains incomplete unless one superimposes some kind
of sufficient conditions.
The philosophy in Step 3 consists in treating sensitivity calculations as
by-products of any solution algorithm for x 0 , u 0 . In control theory, this has
been the primary impetus for the numerical implementations in [4] - [6], [10],
[19], [20]. This approach parallels Fiacco's book [14] where the main theme
is to develop a "sensitivity methodology including software interfacing with
the calculations required by any of the standard NLP codes".
The numerical implementations of Step 3 have been extended to control
problems including control and state inequality constraints by Bock and
Kr-amer-Eis [2], [3], [22], Pesch [40] - [44], Kugelmann and Pesch [23], [24]
and also by Dillon and Tun [10]. It appears that theory lags behind numerical
implementation. SSCs for control problems including inequalities have
not yet been developed to full extent. We have reasons to believe that the
main ideas of this paper carry over to the general case.
sufficient conditions and shooting
methods
We shall begin with the unperturbed problem P (p 0 ) and suppress the argument
0 in this section. We shall summarize the second-order sufficient
conditions (SSCs) derived in [36], [38], [50], [54], [55] and establish the close
connections between SSCs and shooting methods for solving the associated
boundary value problem (BVP).
Let the following data be given: a fixed interval [a; b] ae IR, end-points
x a , x b 2 IR n , an open, convex and bounded set U ae IR m and functions
. The control problem
(P ) is defined to be:
R
a
over all feasible pairs (x; u) of piecewise continuous functions
and absolutely continuous functions x such that
The control constraint (2.3) with U open and bounded has been introduced
for technical reasons. Mainly it should allow for a practical verification of
the regularity condition in Definition 2.1.
Let C n [a; b] denote the space of continous functions x
equipped with the usual topology. For
by B(x; ") the open "-ball around x in C n [a; b]. Similarly, the tube about
is the set
A feasible pair called a (strong) local minimum if for some ffl ? 0
for all feasible pairs (x; u) with x 2 B(x
We shall use hereafter the terminology
function '. Given a pair shall assume the following hypothesis:
functions L and f are of class C k with k - 2 on
linearized system -
completely controllable in
every interval [a; c] for a - c - b.
The controllability assumption (H 2 ) is usually referred to as the normality
condition.
The Hamiltonian of (P ) is defined by
where T denotes the transpose. Assuming normality (H 2 ) the first-order
necessary conditions for a strong local minimum (minimum principle) are as
follows: there exists an absolutely continuous function
that
The latter minimum condition yields
H
One basic assumption for SSCs is that the strengthened Legendre condition
holds:
positive definite for t 2 [a;
This condition is not sufficient to guarantee the continuity of the control
(t). The continuity and, in fact, the smoothness of u 0 (t) follows from the
regularity of the Hamiltonian.
Definition 2.1 Let k - 1. The Hamiltonian H is called
-function
such that
is the unique minimum for all (t; x; -) 2 T
This condition strengthens the regularity condition (1.2) 000 of Zeidan [55],
p. 22. Also, this regularity condition is tacitly underlying numerical methods
for solving the BVP defined by (2.1), (2.2), (2.5) and (2.6). This can be seen
as follows. The optimal solution obtained by solving the BVP
with boundary values . The solutions x 0 (t), - 0 (t) of this
are C k -functions since the right hand side of (2.8) is a C k -function.
Hence the optimal control
is also a C k -function.
Now we shall need the variational system corresponding to (2.8). The
continuity of u 0 (t) and (2.7) imply that there exists " ? 0 such that the
-function u   (t; x; -) in Definition 2.1 satisfies
for (t; x; -) 2 T
By differentiation of the first equation we obtain the identities
and hence in view of the second relation in (2.10) and H
Then the variational system for (2.8) about
where
We shall use the system as well with vector solutions y(t); j(t) as with (n; n)-
matrix solutions y(t); j(t).
Let us indicate the connection between the variational system (2.12) and
shooting methods for solving the BVP (2.2), (2.8). Consider the ODE with
initial values depending on a shooting parameter s 2 IR n (compare [7], [28],
The solutions denoted by x(t; s) and -(t; s) are C k -functions for s near
We have to solve the nonlinear equation
F
where F is a C k -function for s near s 0 . Applying Newton-like procedures
requires the nonsingularity of the matrix
@s
@x
@s
The matrix y(b) is computed by noting that the matrices
@s
@s
are solutions of the variational system (2.12) with initial conditions
From Reid [45], p. 36, and Zeidan, Zezza [56], [57] we infer the following
conjugate point definition. A point c 2 [a; b) is called conjugate to b, if
there exist vector functions y(t); j(t) with y(t) 6j 0 satisfying the variational
system (2.12) with boundary conditions There is
a close connection between matrix solutions y(t), j(t) of (2.12) and matrix
solutions Q(t) of the Riccati differential equation
In fact, if y(t); j(t) are matrix solutions of (2.12) with
det
then the (n; n)-matrix satisfies the Riccati equation (2.19).
The following theorem summarizes the basic results concerning the dis-
conjugacy of the variational system (2.12) and solutions of the Riccati equation
(2.19). The proof immediately follows from Theorem 7.1 in Reid [45],
p. 138; see also [57], Corollaries 6.1 and 6.2.
Theorem 2.1 Assume that the normality hypothesis (H 2 ) and the strengthened
Legendre condition (2.7) hold. Then the following statements are equivalent:
(a) There exists no point c 2 [a; b) which is conjugate to
(b) There exists no point c 2 (a; b] which is conjugate to
(c) The matrix solution y(t), j(t) of (2.12) with initial conditions
b). The matrix solution y(t); j(t) of (2.12) with
(e) The Riccati equation (2.19) admits a finite C 1 -solution Q(t) in [a; b].
Remark Statement (c) of this theorem is known as the Jacobi condition. In
particular, this condition comprises the nonsingularity of the shooting matrix
y(b) in (2.16). Note also that condition (c) is easier to verify numerically than
condition (e); compare the example in Section 4.
The following SSCs follow from [36], Theorem 5.2, [38], Theorem 2.2, [50],
Theorem 5.3 and [55], Theorem 2.2.
Theorem 2.2 Let feasible pair for (P) such that Hypothesis
Assume that there exist an absolutely continuous function
such that the necessary conditions (2.5), (2.6) are satisfied
and assume further that the following conditions hold:
(b) the Hamiltonian H is C k -regular,
(c) there exists a symmetric C 1 -solution Q(t) of the Riccati equation (2.19).
local minimum for (P ) and, moreover, u 0 is a C k -
function.
Note that conditions (a) - (c) are stable with respect to small C k -pertur-
bations of the data. This property is crucial for the second-order sensitivity
result in the next section.
3 Second-order sensitivity
The problem (P ) considered in Section 2 is embedded into the following
parametric control problem P (p) depending on a parameter p 2
Minimize J(x; u;
Z
a
subject to
The unpertubed problem corresponding to identified with
problem (P ) of Section 2. Let
replaced by:
are of class and the
functions are of class C k on B(p
The Hamiltonian for problem P (p) is
We assume that the second-order sufficient conditions of
Theorem 2.2 with a C k -function - 0 . The C k -regularity of the unperturbed
Hamiltonian (2.4) carries over to the perturbed Hamiltonian (3.4): there
exists " ? 0 and a C k -function
such that the minimum of H is uniquely attained at
for all (t; x; -; p) 2 T "). The uniqueness follows from the
compactness of U combined with arguments used in Proposition 3.1 in [55].
The smoothness property of u   is a consequence of the implicit function
theorem since u   satifies
and the strict Legendre condition (2.7) holds.
Now we can state the main result of this paper. A preliminary version
for more general control problems has been announced in [43].
Theorem 3.1 (Second-order sensitivity)
Let feasible for the unperturbed problem P (p 0 ) such that Hypothesis
holds. Assume that the second-order sufficient
conditions of Theorem 2.2.
Then there exists a neighborhood V ae IR k of -functions
such that the following statements hold:
(2) the triple x( satisfies the second-order sufficient
conditions in Theorem 2.2 for all
strong local minimum for P (p).
Proof: In a first step we construct functions x(t; p), u(t; p), -(t; p) satisfying
the first order conditions (2.5), (2.6) for p near p 0 . Using the minimizing
function u   (t; x; -; p) in (3.5), this amounts to solve the BVP
The shooting procedure is a parametric version of (2.14) - (2.18). We consider
the ODE (3.6) with initial values depending on a shooting parameter s 2 IR n
Solutions ~
exist in [a; b] for ksk and small and
are C k -functions with respect to all arguments (t; p; s). Then the mapping
defined by
is a C k -function with F (p Solving the BVP (3.6), (3.7) then is
equivalent to solving the nonlinear equation F (p; In order
to apply the implicit function theorem we have to check the nonsingularity
of the (n; n)-matrix
@s
@s
As we have already seen in (2.16) this matrix agrees with the matrix y(b)
where
@s
@s
are solutions of the variational system (2.12) with initial conditions
Theorem 2.1(c) asserts the nonsingularity of y(b). The implicit function
theorem then yields a neighborhood V ae IR k of -function
The conclusion to this point is that the functions
x(t; p) := ~
x(t;
-(t;
are C k -functions which solve the BVP (3.6) and (3.7) for . The
associated control function
u(t; p) := u   (t; x(t; p); -(t; p))
is also of class C k and satisfies the minimum principle in view of (3.5). Claim
(1) of the theorem is immediate.
In a second step we have to show that, indeed, x(t; p) and u(t; p) are
optimal for problem P (p). We can choose the neighborhood V so small that
the following two statements are true for all
(a) the strict Lengendre condition holds
(b) the Riccati equation
has a symmetric C 1 -solution Q(t; p) on [a; b] where A(t; p), B(t; p),
C(t; p) are the matrices (2.13) evaluated at x(t; p), -(t; p), u(t; p).
The last statement (b) follows from the standard embedding theorem for
ODE. Applying Theorem 2.2 for each we arrive at the desired conclusion
that the pair (x( is a local minimum for every shall briefly illustrate now the use of this sensitivity result when devising
efficient numerical feedback schemes for neighboring extremals. Since
the functions x(t; p), -(t; p) and u(t; p) are of class C k on [a; b] \Theta V (k - 2)
the following Taylor-expansions exist:
The "variations"
are (n; p) resp. (m; p) matrices of class C 1 which satisfy the linear inhomogeneous
(3.
Here the upper index zero denotes arguments evaluated at This
result follows by differentiating (3.6) - (3.8) with respect to p. Moreover, the
differentiation of the identity
H u (t; x(t; p); -(t; p); u(t; p);
yields
The linear BVP (3.9), (3.10) can be solved by stable shooting techniques;
(see [2], [3], [22] - [24], [40] - [43]).
A further consequence of Theorem 3.1 is that the optimal value function
is a C k -function in V . It can be seen from formulas (2.8) and (4.8) in [35]
that the differential is given by
Z
a
In case that represents the only perturbation, this yields the
well known shadow-price formula J 0 (p
4 An illustrative example
We present an example which admits two kinds of extremal solutions both
with a nonsingular shooting matrix. The sufficient conditions single out only
one solution as optimal. For this solution a sensitivity analysis is performed
according to Theorem 3.1. Consider the following variational problem depending
on a parameter p 2 IR:
subject to
The unperturbed problem corresponds to p as usual the
control variable by u := -
x the Hamiltonian becomes
The strict Legendre condition H throughout. The function
minimizing the Hamiltonian is u   (x; -; \Gamma-. The Hamiltonian
(4.1) is C 1 -regular. The BVP (3.8) is given by
Unperturbed solution for
Using shooting methods, Stoer and Bulirsch [51], p. 170, have shown that
the BVP (4.2) with has two solutions x 0
characterized by
The two solutions are shown in Figure 1.
Figur 1: Solutions x 0 (t) and x 1 (t) of BVP (4.2);
conjugate point t
In order to test x 0 (t) and x 1 (t) for optimality we have to check the Jacobi
condition in Theorem 2.1 (c). The variational equation for (4.2) with respect
to x 0 (t) or x 1 (t) is (compare also (2.12) with boundary conditions (2.18)):
It can be verified by numerical integration that the Jacobi condition holds:
Hence x 0 (t) is optimal since the second-order sufficient optimality conditions
in Theorem 2.2 are satisfied. On the other hand, numerical integration shows
that
This means that the point t c 2 (0; 1) is conjugate to
the necessary condition of optimality in [56], Theorem 6.1. Hence x 1 (t) is
non-optimal. We note that the exact value of the conjugate point t c can be
computed via the BVP (4.4) and y 1 treating t c as a free variable.
Perturbed solutions and neighboring extremals:
By Theorem 2.2 there exists a neighborhood V ae IR of p
function x(t; p) for (t; p) 2 [0; 1] \Theta V such that x( \Delta ; p) is optimal for the
variational problem and satisfies x(t;
The function x(t; p) solves the BVP (4.2) and admits a Taylor-expansion
on [0; 1] \Theta V . The variations
are solutions of the linear inhomogeneous BVP
resp.
which can be obtained from (4.2) by formal differentiation; compare also
(3.9), (3.10). The solutions z 1 (t) and z 2 (t) are given by

Table

1: First- and second-order Taylor approximation in (4.5),

Table

presents some numerical results reflecting the error of the first- and
second-order Taylor order expansion in (4.5) where 2:
5 Conclusion
The second-order sensitivity result derived in this paper states that the optimal
solution of a nonlinear control problem is differentiable with respect
to parameters provided that the second-order sufficient conditions (SSCs)
hold for the unperturbed (nominal) problem. It is shown that SSCs include
the nonsingularity of the shooting matrix for the associated boundary value
problem. Many authors have used the nonsingularity of the shooting matrix
as the only tool to obtain a differentiable family of extremals. The example
in section 4 demonstrates that this alone does not suffice to find an optimal
solution to which perturbation analysis can be applied. Thus, the solution
differentiability result in this paper gives a firm theoretical basis to existing
numerical feedback schemes for computing neighboring extremals.
It is desirable to extend the solution differentiability to perturbed non-linear
control problems with inequality constraints of the type
mixed state-control constraints: C(x; u; p) -
state constraints: S(x; p) - 0
The main obstacle to extend the techniques of this paper to such inequality
constraints is the fact that SSCs in [36], [50] are too strong and are not
directly related to the variational system of the boundary value problem.
SSCs which use a type of Riccati ODE modelled after this variational system
have been obtained in [38] for the special constraint C(u) - 0. It will be
our future concern to generalize both the existing SSCs and the second-order
sensitivity result to problems with inequality constraints.



--R

Stability of Solutions to Control Constrained Nonlinear Control Problems
Zur numerischen Behandlung zustandsbeschr-ankter Steu- erungsprobleme mit Mehrzielmethode und Homotopieverfahren

On the Conjugate Point Condition for the Control Problem
Optimization and Control of Nonlinear Systems Using the Second Variation
Applied Optimal Control
Die Mehrzielmethode zur numerischen L-osung von nichtlinearen Randwertproblemen und Aufgaben der optimalen Steue- rung
"Anwendungsbezogene Optimierung und Steuerung"

Application of Sensitivity Methods to the Problem of Optimal Control of Hydrothermal Power Systems

On Sensitivity in Optimal Control Systems
Nonlinear Programming: Sequential Unconstrained Minimization Techniques
Introduction to Sensitivity and Stability Analysis in Non-linear Programming
Directional Behaviour of Optimal Solutions in Nonlinear Mathematical Programming Problems
Necessary and Sufficient Conditions for a Local Minimum: Second Order Conditions and Augmented Duality
Solution Point Differentialbility without Strict Complementarity in Nonlinear Programming
Implicit functions and sensitivity of stationary points
Guidance Theory and Extremal Fields
An Optimal Guidance Approximation Theory
Strongly Stable Stationary Solutions in Nonlinear Pro- grams

A New General Guidance Method in Constrained Optimal Control
A New General Guidance Method in Constrained Optimal Control
Sensitivity Analysis for Nonlinear Programs and Variational Inequalities with Nonunique Multipliers
Solution Differentiability for Variational Inequalities
Optimal Trajectory
The Close Relationship Between Methods for Solving Two-Point Boundary Value Problems
Differential Stability of Solutions to Convex
On Differentiability with Respect to a Parameter of Solutions to Convex Optimal Control Problems Subject to State Space Constraints
Stability and Sensitivity of Solutions to Optimal Control Problems for Systems with Control Appearing Linearly
Senisitivity Analysis of Optimization Problems in Hilbert Space with Application to Optimal Control
Sensitivity of Solutions to Convex
Numerical Solution of Singular Control Problems Using Multiple Shooting Techniques
Differential Stability in Optimal Control Problems
First and Second Order Sufficient Optimality Conditions in Mathematical Programming and Optimal Control
Berechnung optimaler Steuerungen von Heizung und K-uhlung f?r ein realistisches Sonnenhausmodell
Another Jacobi Sufficiency Criterion for Optimal Control with Smooth Constraints
Sensitivity of the Performance of Optimal Control Systems to Plant Parameter Variations
Echtzeitberechnung fastoptimaler R-uckkopplungssteuerun- gen bei Steuerungsproblemen mit Beschr-ankungen


Optimal Control Problems under Disturbances
Optimal and Nearly Optimal Guidance by Multiple Shoot- ing
Riccati Differential Equations
Strongly Regular Generalized Equations

Second Order Sensitivity Analysis and Asymptotic Theory of Parametrized Nonlinear Programs
Sensitivity Analysis of Nonlinear Programs and Differentiability Properties of Metric Projections
Sufficient Optimality Conditions for Nonconvex Problems with State Constraints
Introduction to Numerical Analysis.
Projection on a Cone
On the Sensitivity of Optimal Control Systems
Sufficient Conditions for the Generalized Problem of Bolza
Sufficiency Conditions with Minimal Regularity Assump- tion
Necessary Conditions for Optimal Control Problems: Conjugate points
The Conjugate Point Condition for Smooth Control Sets
--TR
parametric control problems;solution differentiability;feedback controls;shooting methods;second-order sufficient conditions