Finite Horizon Density Steering for Multi-input State Feedback Linearizable Systems
Kenneth F. Caluya Abhishek Halder


Extreme Spin Squeezing from Deep Reinforcement Learning
    Li You
    December 30, 2023
=======================================================

empty
empty


#1#1


1





December 30, 2023

Multi-Input Feedback Linearization
    Kenneth F. Caluya, Abhishek Halder
    December 30, 2023
======================================






In this paper, we study the feedback synthesis problem for steering the joint state density or ensemble subject to multi-input state feedback linearizable dynamics. This problem is of interest to many practical applications including that of dynamically shaping a robotic swarm. Our results here show that it is possible to exploit the structural nonlinearities to derive the feedback controllers steering the joint density from a prescribed shape to another while minimizing the expected control effort to do so. The developments herein build on our previous work, and extend the theory of the Schrödinger bridge problem subject to feedback linearizable dynamics. 







§ INTRODUCTION

We consider the problem of steering the statistics of the state vector (t) from a prescribed ensemble or joint density ρ_0() to another ρ_1() over a finite time horizon t∈[0,1], subject to controlled nonlinear dynamics of the form 

    = () + G(),   ∈𝒳⊆ℝ^n,   ∈ℝ^m,

where  is a smooth vector field on the state space 𝒳⊆ℝ^n, and G is an n× m matrix whose columns consist of the vectors _i∈ℝ^n for i=1,…,m, i.e.,

    G() = [ _1()|_2()|…|_m
        ()].

It is of broad practical interest to solve this finite horizon density steering problem while minimizing the average total control effort over the controlled state ensemble ρ(,t). 

This problem is motivated by the growing need across science and engineering applications to control a large population of systems. Consider for example, shaping the bulk magnetization distribution for Nuclear Magnetic Resonance spectroscopy, controlling heterogeneous (e.g., aerial and ground) robotic swarms <cit.>, strategically synchronizing and desynchronizing a neuronal population to regulate the Parkinsonian tremor <cit.>, and differentially moving the setpoints of a large population of residential air-conditioners by a service provider to make their total energy consumption track the intermittency in supply (e.g., due to stochastic renewable generation) in a privacy-preserving manner <cit.>. These exemplars concern population ensemble or density whose shape is actively controlled over time while preserving the physical mass. The conservation of mass allows an alternative interpretation of the underlying mathematical problem – instead of steering a large number of dynamical systems, one can think of steering a single system with probabilistic uncertainty in its initial and terminal state, i.e., ρ_0,ρ_1 being joint state probability density functions (PDFs). This too, arises naturally in practice, e.g., in robot motion planning <cit.>, where uncertainties in the initial and terminal states are unavoidable due to process and sensor noise. 


From a system-control theoretic viewpoint, finite horizon density steering via feedback is a non-classical stochastic optimal control problem. The qualifier “non-classical" points to the fact that finding the feedback policy requires solving an infinite dimensional two-point boundary value problem on the manifold of joint state PDFs. This is an emerging research direction in the systems-control community wherein recent advances <cit.> have uncovered its connections with the theory of optimal mass transport <cit.> and the Schrödinger Bridge Problem (SBP) <cit.>. Also, there have been results on the covariance steering problem <cit.> which concerns steering second order state statistics. With the exception of <cit.>, almost all works have focused on steering the state statistics over a controlled linear system. 

In this paper, we consider finite horizon density steering state feedback linearizable systems of the form (<ref>). The nonlinearities in (<ref>) induce non-Gaussian statistics even if the endpoint PDFs are both jointly Gaussian. Thus, finding the feedback solution of the density steering problem in a non-parametric sense, is non-trivial. The main contribution of this paper is to show that it is possible to exploit the feedback linearizing transformation for density steering. In particular, we obtain the optimal state feedback policy in terms of the solution of certain Hamilton-Jacobi-Bellman (HJB) partial differential equation (PDE). Furthermore, we show that a dynamic stochastic regularization can be used to derive a system of boundary-coupled linear PDEs, which we refer to as the Schrödinger system, whose solutions recover the optimal state feedback and the optimal controlled joint state PDF. We envision that the theoretical developments herein will help design algorithms solving the feedback density steering over nonlinear dynamical systems.




 §.§ Notations and Preliminaries

Throughout the paper, we will use bold-faced
capital letters for matrices and bold-faced lower-case letters
for column vectors. We use the symbol ⟨· , ·⟩ to denote the Euclidean inner product. In particular, ⟨, ⟩ := trace(^⊤)
denotes Frobenius inner product between matrices  and
, and ⟨a, b⟩  := a^⊤b denotes the inner product between
column vectors a and b.
We use 0 and 1 to denote the vector consisting of all zeros, and all ones, respectively, and the symbol e_k to denote the kth  standard basis vector of appropriate dimension. We use ∇_, ∇_·, and Hess(·) to respectively denote the Euclidean gradient, divergence and Hessian operators w.r.t. vector . The Lie bracket of two vector fields ξ and η at ∈ℝ^n is a new vector field [ξ,η]():=(∇_η)ξ() - (∇_ξ)η(). For k∈ℕ, the k-fold Lie bracketing of η with the same vector field ξ is denoted as ad_ξ^kη:=[ξ,ad_ξ^k-1η]; by convention ad_ξ^0η=η. The Lie derivative of a scalar-valued function λ() w.r.t. the vector field ξ evaluated at  is L_ξλ():=⟨∇_λ,ξ⟩(). For k∈ℕ, the k-fold Lie derivative of h w.r.t. the same vector field ξ evaluated at  is denoted as L_ξ^kλ():=⟨∇_L_ξ^k-1λ,ξ⟩(); by convention L_ξ^0λ()=λ(). Given m vectors fields ξ_1(), , ξ_m() in ℝ^n, we say 𝒟():=span{ξ_1, , ξ_m}() is involutive at ∈ℝ^n, if for all ξ_i(),ξ_j()∈𝒟(), we get that the Lie bracket [ξ_i,ξ_j]()∈𝒟(), where i,j=1,,m. The notation ∼ρ denotes that the random vector  has joint PDF ρ. Furthermore, we denote the pushforward of a PDF by the symbol ♯. We use the symbol ∘ to denote function composition, and (·) to denote the support of a function.








§ MIMO FEEDBACK LINEARIZATION

We consider multiple-input control system of the form (<ref>), and recall some well-known results on feedback linearization <cit.> that will be useful in the sequel.
 (Full state static feedback linearization)
System (<ref>) is said to be full state static feedback linearizable around a point _0 ∈𝒳 if there exists a smooth feedback of the form = () + () defined on 𝒳, and a  diffeomorphism :𝒳↦ℝ^n such that the change of variables := () transforms (<ref>) into 

    =  + ,   ∈ℝ^n,   ∈ℝ^m,

wherein the pair (,) is controllable.

 In other words, (<ref>) is full state static feedback linearizable if there exists a triple ((),(),()) such that 

    (∇(()+G()() ) )_=^-1()   = , 
    (∇(G()() ) )_=^-1()   = ,

where the pair (,) satisfies

    rank[,,^2,…^n-1] = n.
 
<cit.> (Vector relative degree)
Consider the Multi-Input Multi-Output (MIMO) system:

    = () + () , 
       = h(),

where ∈𝒳⊆ℝ^n, ∈ℝ^m as before. Furthermore, h():=(h_1(),h_2(),… h_m() ) ∈ℝ^m, where h_j are smooth scalar-valued functions for all j=1…,m. The input-output system (<ref>) is said to have vector relative degree π = (π_1,π_2,…,π_m) at _0 ∈𝒳, if

    L__jL^k_h_i() ≡ 0,    1 ≤ i,j≤ m,    1≤ k<π_i-1,
 
and the m× m matrix

    () := [ L__1L^π_1-1_ h_1()                  … L__mL^π_1-1_ h_1();                  ⋮                  ⋱                  ⋮; L__1L^π_m-1_ h_m()                  … L__mL^π_m-1_ h_m() ]

evaluated at =_0, is non-singular. 


Here, π_j∈ℕ, j=1,,m, is the number of times one has to differentiate the jth output y_j w.r.t. t such that at least one of the m input components appears explicitly in the expression for y_j^(π_j). In other words, π_j is the number of integrators between the input and the jth output.


 
It is known <cit.> that given an n-dimensional vector field , and a matrix () of rank m, the system (<ref>) is full state static feedback linearizable if and only if:

(i) there exist functions h_1(), h_2(),…,h_m(), such that the input-output system (<ref>) has relative degree  at _0∈𝒳,

and
 
(ii) the relative degree  is such that π_1+π_2+… + π _m =n, where n is the dimension of the state vector .

The output function () play an  important role in transforming (<ref>) into a controllable linear system. If we can find (·) satisfying conditions (i)-(ii) in Remark <ref>, then we can use the same to construct a state feedback law and a desired change of coordinates. Explicitly, this feedback law can be obtained as 

    = -(())^-1d()_:=()+ (())^-1_:=(),

where () is as in (<ref>), and 

    d():= (L^π_1_h_1(), L^π_2_h_2(), …, L^π_m_h_m())^⊤,

The linearizing coordinates :=  () are subdivided as

    = [ ^1; ^2;  ⋮; ^m ],   () = [  ^1(); ^2 ();     ⋮; ^m () ],

where each ^i,^i∈ℝ^π_i, i=1,,m,  have  components 

    z^i_k = τ^i_k():=L_^k-1h_i(),    k=1,,π_i.

The feedback law (<ref>) and the change of coordinates (<ref>), together transform (<ref>) which is in state-control pair (,), into the
Brunovsky canonical form in the state-control pair (,), given by

    =  + ,   ∈ℝ^n,   ∈ℝ^m,

where , are block diagonal matrices

    := diag(_1,_2, …,_m),   := diag(b_1,b_2,…,b_m),
 
wherein for each i=1,,m, we have 

    _i := [0|e_1|e_2|…|e_π_i-1] ∈ℝ^π_i×π_i,   b_i:= e_π_i∈ℝ^π_i.



Since static state feedback linearization is equivalent to Remark <ref>, hence the matrix (<ref>) is invertible at =_0. This guarantees that () and () in (<ref>) are well-defined at _0.

As seen above, the existence of the (fictitious) output (·) is a necessary and sufficient condition for full state feedback linearization. The following result allows us to establish the existence of the (·) under suitable conditions on the vector fields (),_1(),…, _m(). Thus, the  conditions for full state feedback linearization can be restated as the following.

<cit.>
Consider the system (<ref>) where rank((_0))= m, and for i = 0,1,…, n-1, let 
    
    Δ_i() :=  span{ad_^k_k: 0 ≤ k ≤ i,1≤ j ≤ m}.

Then, there exist scalar-valued functions h_1(),…, h_m(), defined on 𝒳 such that (<ref>) has relative degree π at _0, with π_1+ π_2 + … + π_m = n, iff:

(i) Δ_i has constant dimension near _0 for each i=0,1,,n-1,

(ii) Δ_n-1 has dimension n,

(iii) Δ_i is involutive for each i=0,1,,n-2.

Proposition <ref> helps to verify if a given system of the form (<ref>) is full-state feedback linearizable. For the construction of the functions h_i in Proposition 1, we refer the readers to <cit.>. 

Let us consider a system of the form (<ref>) defined on a neighborhood of _0=0, given by

    = [           x_2+x_2^2;   x_3-x_1x_4+x_4x_5; x_2x_4+x_1x_5-x_5^2;                 x_5;               x_2^2 ]_() +[            0;            0; cos(x_1-x_5);            0;            0 ]__1()u_1 
                + [ 1; 0; 1; 0; 1 ]__2() u_2.

A direct computation verifies that (<ref>) satisfies the conditions (i)-(iii) of Proposition <ref>, implying the existence of output functions h_1(),h_2(). Following the constructive steps in <cit.>, these output functions can be obtained as

    y_1 =h_1= x_1 -x_5, y_2 =h_2=x_4.


Notice that 

    L__1 h_1()    ≡ L__2 h_1() ≡ L__1L_ h_1() ≡ L__2L_ h_1() ≡ 0,
     
        L__1 h_2()    ≡  L__2 h_2() ≡ 0,

and that the matrix

    (_0)    =
    [ L__1L_^2 h_1() L__2L_^2 h_1();   L__1L_ h_2()   L__2L_ h_2() ]|_=_0
       = 
    [ cos(x_1-x_5)            1;            0            1 ] |_=_0 =[ 1 1; 0 1 ],

is non-singular. Therefore, (<ref>) with output (<ref>) has vector relative degree =(π_1,π_2)= (3,2) satisfying π_1+π_2 = 3+2 =5, which is indeed the dimension of the state space. From (<ref>), we obtain the change of coordinates

    ():= [      h_1();    L_h_1(); L_^2 h_1();      h_2();   L_ h_2() ]= [            x_1-x_5;                x_2; x_3 -x_1x_4+x_4x_5;                x_4;                x_5 ].

In this case, (<ref>) and (<ref>) yields

    d()    = [  L^3_ h_1(); L^2_ h _2() ] = [     0; x_2^2 ], 
     
        (())^-1   = [  1/cos(x_1-x_5) -1/cos(x_1-x_5);               0               1 ],

which, following (<ref>), result in the feedback law  

    = [ - x_2^2/cos(x_1-x_5);                x_2^2 ]_()
       + 
    [  1/cos(x_1-x_5) -1 cos(x_1-x_5);               0               1 ]_().

Hence, we have constructed a triple ((),(),τ()) given by (<ref>)-(<ref>), that transform (<ref>) into

    =[ 0 1 0 0 0; 0 0 1 0 0; 0 0 0 0 0; 0 0 0 0 1; 0 0 0 0 0;   ]_ +
    [ 0 0; 0 0; 1 0; 0 0; 0 1;   ]_.











§ MINIMUM ENERGY DENSITY CONTROL




 §.§ Stochastic Optimal Control Problem

Given system (<ref>), and two prescribed endpoint PDFs ρ_0(),ρ_1(), we consider the following minimum energy finite horizon stochastic optimal control problem:


    ∈𝒰inf      𝔼{∫_0^11/2‖(x,t)  ‖_2^2  t }, 
          subject to       = () + () , 
             (0) ∼ρ_0()   (1) ∼ρ_1(),


where the state space is 𝒳⊆ℝ^n, ∈ℝ^m and  (<ref>) is feedback linearizable. The infimum is taken over the set of admissible controls with finite energy, i.e.,  𝒰:={:ℝ^n × [0,1]↦ℝ^m | ‖‖_2^2 < ∞}, and the expectation operator 𝔼{·} in (<ref>) is w.r.t. the controlled joint state PDF ρ(,t) satisfying endpoint conditions (<ref>). The objective is to steer the joint PDF ρ(,t) from the given initial PDF ρ_0 at t=0 to a terminal PDF ρ_1 at t=1 while minimizing the expected control effort. 

The problem (<ref>) can be recast into a “fluid dynamics" version <cit.>, which is the following variational problem:
 

    ρ,inf      ∫_0^1∫_𝒳1/2‖(,t) ‖_2^2 ρ(,t)     t, 
       subject to      ∂ρ/∂ t + ∇_·(ρ ((
    ) + ()  )) =0, 
             ρ(,t=0) = ρ_0,   ρ(,t=1) = ρ_1.


Here, the infimum is taken over 𝒫(𝒳) ×𝒰, where 𝒫(𝒳) denotes the space of all joint PDFs supported on 𝒳. We note that (<ref>) is the Liouville PDE <cit.> associated to the dynamical system (<ref>).






 §.§ Reformulation in Feedback Linearized Coordinates

In our recent work <cit.>, we considered the problem (<ref>) for the single-input case, i.e., the case ()≡ [ _1()]∈ℝ^n, and the input u is scalar-valued. The main idea in <cit.> was to recast (<ref>), which is in state-control pair (,), into an equivalent formulation in feedback linearized state-control pair (,). This was made possible by using the diffeomorphism : 𝒳↦𝒵 to pushforward the endpoint PDFs ρ_0,ρ_1 to PDFs σ_0,σ_1 supported on the feedback linearized state space 𝒵. Specifically,

    σ_i():= _♯ρ_i = ρ_i(^-1())|det(∇__=τ^-1() |,    i ∈{0,1},

and 𝒵 := {∈ℝ^n|  = τ(), ∈𝒳}.

Since  is a diffeomorphism, 

the PDFs {σ_i}_i=0,1 supported on the feedback linearized state space 𝒵, are well defined, i.e., spt(σ_i) ⊆𝒵
provided that spt(ρ_i)⊆𝒳.

To generalize the reformulation in <cit.> for the multi-input case, we proceed by setting

    _ : = ∘ ^-1,   _: = ∘ ^-1,

where δ and Γ are as in (<ref>). Using () = _τ() + _(), we now transcribe (<ref>) into
 

    σ,inf      ∫_0^1∫_𝒵1/2ℒ(,)  σ(,t)     t,
       subject to      ∂σ/∂ t + ∇_·( ( +  )σ) =0, 
             σ(,t=0) = σ_0,   σ(,t=1) = σ_1,


where

    ℒ(,):= ‖_() + _() ‖_2^2.

The infimum in (<ref>) is taken over the pair of transformed PDFs and admissible controls (σ,) ∈𝒫(𝒵) ×𝒱 where 𝒱:={v: 𝒵× [0,1] ↦ℝ^m| ‖ v ‖_2^2 < ∞}. 


 
The solution pair (ρ^opt,^opt) for (<ref>) can be recovered from the optimal solution (σ^opt,^opt) of (<ref>) via the transformations 

    
    ρ^opt(,t)   = σ^opt((),t) |det∇__() |, 
    ^opt(,t)   = () +  ()   ^opt(^-1(),t)

    

for ∈𝒳, and t∈ [0,1]. 



 
To illustrate the reformulation (<ref>), let us reconsider the system (<ref>). In this case, the inverse mapping of (<ref>) is given by

    = ^-1():= [      z_1+z_5          z_2 z_3 + z_1z_4          z_4          z_5 ]^⊤.
 
Here, the determinant of the Jacobian of (<ref>) is non-zero for all vectors in ℝ^n, i.e., 𝒵=ℝ^n.

From (<ref>) and (<ref>), we have 

    _() = [ -z_2^2/cos(z_1);           z_2^2 ],

and 

    _() = [  1/cos(z_1) -1/cos(z_1);           0           1 ].

The functional ℒ(,) in (<ref>) equals 

    ^⊤[ 2/cos^2(z_1)  -1/cos(z_1);  -1/cos(z_1)            1 ] + ⟨(z_2^2/cos^2(z_1), .
       .-z_2^2/cos(z_1) + z_2^2
                           )^⊤, ⟩ + z_2^4/cos^2(z_1) + z_2^4.



 Because feedback linearization guarantees that the matrix pair (,) is controllable, any vector _1 ∈𝒵 is reachable from any other vector _0 ∈𝒵 for all t∈[0,1] via the flow of (<ref>). This ensures that in (<ref>), the initial PDF σ_0() can be steered to σ_1() via the flow σ(,t) of the controlled Liouville PDE (<ref>). Thus, the constraint set of (<ref>) is non-empty, and the problem is feasible.





 §.§ Optimality
 
To show the existence and uniqueness of minimizer for (<ref>), we set m
:=σ, and consider the change of variable (σ,) ↦ (σ,m), transforming (<ref>) into 
 

    σ,minf      ∫_0^1∫_𝒵𝒥(σ,m )    t,
       subject to      ∂σ/∂ t + ∇_·( σ+ m ) =0, 
             σ(,t=0) = σ_0,   σ(,t=1) = σ_1,


where 

    𝒥(σ,m):=
    1/2‖_() + _() m/σ‖_2^2 σ   if σ >0,
    
    0    if (σ,m) = (0,0), 
    
    +∞   otherwise.

We note that 𝒥(σ,m) is the perspective function of the strictly convex map m↦‖_()σ + _() m‖_2^2; therefore, 𝒥 is jointly strictly convex in (σ,m). The constraints (<ref>)-(<ref>) are linear in (σ,m). Hence, (<ref>) admits a unique minimizing pair, and equivalently, so does (<ref>). The following theorem summarizes how this optimal pair for (<ref>), denoted hereafter as (,), can be obtained.


 
(Optimal control for (<ref>))
The optimal control  for the problem (<ref>), is given by

    ^opt(,t) = (_^⊤_())^-1^⊤∇_ψ - ^-1_() _(),


where ψ solves the Hamilton-Jacobi-Bellman (HJB) PDE 

    ∂ψ/∂ t+⟨∇_ψ, ⟩-⟨∇_ψ,^-1_()_() ⟩
       +  1/2⟨∇_ψ,  (_^⊤() _() )^-1^⊤∇_ψ⟩ = 0.

Furthermore, if the optimal joint state PDF σ^opt is a solution to the Liouville PDE

    ∂σ^opt/∂ t + ∇_·( ( + ^opt)) = 0,

with boundary conditions (,0) =  _0(), and (,1) =  _1(), then the pair (,) solves the problem (<ref>).




The Lagrangian associated with (<ref>) is 

    ℒ(σ,ψ,)    =∫_0^1∫_𝒵1/2ℒ(,) σ(,t)   t 
       + ∫_𝒵∫_0^1ψ(,t)  ∂σ/∂ t t  _term   1
       +∫_0^1∫_𝒵ψ(,t) ∇_·( ( + )σ)   t_term  2.

In (<ref>), we interchange the order of integration and  perform integration by parts w.r.t. t in term 1, and w.r.t.  in term 2. Since σ(,t) → 0 as z →∂𝒵, we can express ℒ as 

    ∫_0^1∫_𝒵{1/2ℒ(,) - ∂ψ/∂ t- 
        ⟨∇_ψ,  + ⟩}σ(,t)  t.

Performing pointwise minimization of the above w.r.t.  while fixing σ, we obtain 

    _^⊤_^opt(,t) = ^⊤∇_zψ(,t) -_^⊤() _().

Taking the matrix inverse on both sides yield (<ref>). Substituting ^opt back into (<ref>) and equating to zero, we then get 

    ∫_0^1∫_𝒵{
          -∂ψ/∂ t- ⟨∇_ψ, ⟩+ ⟨∇_ψ,^-1_()_() ⟩
       - 1/2⟨∇_ψ,  (_^⊤() _() )^-1^⊤∇_ψ⟩}σ(,t) t = 0.

Since (<ref>) holds for arbitrary σ, we arrive at (<ref>). 




 (HJB for (<ref>)) 
From (<ref>)-(<ref>), we have  

    (^⊤_()_())^-1     = [ cos^2(z_1)   cos(z_1);   cos(z_1)          2 ]

 and
 
    _^-1()_() = [     0; z_2^2 ].

Substituting (<ref>)-(<ref>) into (<ref>), and using the pair (,) from (<ref>), gives the HJB PDE 

    ∂ψ/∂ t 
        +z_2 ∂ψ/∂ z_1 +z_3∂ψ/∂ z_2 + z_5∂ψ/∂ z_4 
         - z_2^2 ∂ψ/∂ z_5
       +1/2[cos^2(z_1)(∂ψ/∂ z_3)^2+  cos(z_1)∂ψ/∂ z_3∂ψ/∂ z_5 + 2(∂ψ/∂ z_5)^2] = 0.




Computing the pair (σ^opt,^opt) in Theorem <ref> is challenging in general since it calls for solving a system of coupled nonlinear PDEs (<ref>)-(<ref>) with atypical boundary conditions. In the following Sections, we will provide further reformulations of (<ref>) to make it computationally amenable. 	











§ STOCHASTIC DENSITY STEERING: REFORMULATION INTO SCHRÖDINGER SYSTEM


Motivated by <cit.>, we consider a generalized version of (<ref>) by adding a diffusion term to (<ref>):
 

    σ,inf      ∫_0^1∫_𝒵1/2ℒ(,) σ(,t)     t,
       subject to      ∂σ/∂ t + ∇_·( ( +  )σ)  
                =ϵ1^⊤(D() ⊙Hess(σ) ) 1,
             σ(,t=0) = σ_0,   σ(,t=1) = σ_1,


where ():= _τ ^-1() (_τ^-1())^⊤. In particular, the controlled Liouville PDE in (<ref>) is now replaced by a Fokker-Planck-Kolmogorov PDE in (<ref>), having an additional diffusion term √(2ϵ)_τ ^-1(), where the parameter ϵ>0 (not necessarily small). Formally, this generalization is equivalent to adding a stochastic perturbation to the controlled sample path ODE =  +, resulting in the Itô SDE

    = ( + )   t+ √(2ϵ)_τ ^-1()  w,

where w(t)∈ℝ^m is standard Wiener process. In the special case _τ()≡ 0 and _τ()≡I, problem (<ref>) reduces to the Schrödinger bridge problem with linear prior dynamics <cit.>. Thus, (<ref>) is a Schrödinger bridge-like problem with a prior dynamics that has linear drift and nonlinear diffusion coefficient.

The following Theorem characterizes the minimizing pair (,) for problem (<ref>).



 
(Optimal control for (<ref>))
The optimal control ^opt(,t) for (<ref>) is given by (<ref>), where ψ solves the HJB PDE 

    ∂ψ/∂ t+⟨∇_ψ, ⟩-⟨∇_ψ,^-1_()_() ⟩
       +  1/2⟨∇_ψ,  D() ∇_ψ⟩  + ϵ⟨D(), Hess(ψ) ⟩ = 0,

and the optimal joint state PDF σ^opt(,t) solves the controlled Fokker-Planck-Kolmogorov PDE 

    ∂σ^opt/∂ t   + ∇_·( ( + ^opt )σ^opt)  
       -ϵ1^⊤(D() ⊙Hess(σ^opt) ) 1  = 0,

with boundary conditions 

    σ^opt(,t=0) = σ^opt_0,   σ^opt(,t=1) = σ^opt_1.





The proof proceeds similarly as in Theorem <ref> except that we now have an additional term in the Lagrangian (<ref>) which we refer to as “term 3", given by

    -ϵ∫_0^1∫_𝒵ψ(,t)  1^⊤(D() ⊙Hess(σ^opt) ) 1 t._term   3

From the following chain of equalities:

    ∫_𝒵⟨D(), Hess(ψ)  ⟩σ^opt(,t) 
       = ∫_𝒵∑_i,j=1^n D_ij() ∂ψ(,t)∂ z_i ∂ z_jσ^opt(,t)
       = ∑_i,j=1^n ∫_𝒵D_ij() ∂ψ(,t)∂ z_i ∂ z_jσ^opt(,t)
       = -∑_i,j=1^n ∫_𝒵∂ψ(,t) /∂ z_j∂ (D_ijσ^opt(,t) ) /∂ z_i
       =  ∫_𝒵ψ(,t) ∑_i,j=1^n ∂ (D()_ijσ^opt(,t))∂ z_j∂ z_i
       = ∫_𝒵ψ(,t)  1^⊤(D() ⊙Hess(σ^opt) ) 1

we deduce that (<ref>) is equal to 

    -ϵ∫_0^1∫_𝒵⟨D(), Hess(ψ)  ⟩σ^opt(,t)   t.

So, the expression inside the curly braces in (<ref>), now will have an additional term -ϵ⟨D(), Hess(ψ) ⟩ that is independent of . Therefore, pointwise  minimization of (<ref>) with this additional term
w.r.t. , gives (<ref>), and the associated HJB PDE becomes (<ref>). 

Next, we show that the so-called Hopf-Cole transform <cit.> allows to reduce the system of nonlinear PDEs (<ref>)-(<ref>) with boundary conditions (<ref>), into a system of boundary-coupled linear PDEs, which we refer as the “Schrödinger System".

  
 
 (Schrödinger System) Consider the Hopf-Cole transformation (σ^opt,ψ) ↦ (φ,φ̂): 
  
     
    φ(,t)   := exp(ψ(,t) /2ϵ), 
    φ̂(,t)    := σ^opt(,t) exp(-ψ(,t)/2ϵ),

 
 applied to the system of coupled nonlinear PDEs (<ref>)-(<ref>). The pair (φ,φ̂) satisfies the following system of linear PDEs:
  
 
    ∂φ/∂ t   + ⟨∇_φ,  - _^-1_() ⟩ +  ϵ⟨D,Hess(φ)⟩ = 0,  
    ∂φ̂/∂ t   + ∇_·( (  - _^-1_() )φ̂) 
                -ϵ1( D() ⊙Hess(φ̂)) 1=0,

  
with coupled boundary conditions 

    φ_0()φ̂_0() = σ^opt_0(),  φ_1()φ̂_1() = σ^opt_1().






From (<ref>), ψ = 2ϵlogφ, which yields 

    ∂ψ/∂ t =  2ϵ/φ∂φ/∂ t,   ∇_ψ = 2 ϵ/φ∇_φ.

On the other hand, notice that  

    ϵ⟨D() , Hess(ψ) ⟩
       =   ϵ⟨D() , Hess(2ϵlogφ )⟩
       = 2ϵ^2 ∑_i,j=1^nD_ij() ∂^2logφ/∂ z_i ∂ z_j
       = 2ϵ^2 ∑_i,j=1^nD_ij() ( 1/φ∂^2 φ/∂ z_i z_j - 1/φ^2∂φ/∂ z_i∂φ/∂ z_j) 
       = 2ϵ^2/φ⟨D(), Hess(φ) ⟩  - 2 ϵ^2/φ^2⟨∇_φ,D() ∇_φ⟩.

Substituting (<ref>) and (<ref>) into (<ref>) yields 

    2ϵ/φ∂φ/∂ t + 2 ϵ/φ⟨∇_φ, - _^-1() _() ⟩
        +1/24ϵ^2/φ^2⟨∇_φ, D() ∇_φ⟩ +2ϵ^2/φ⟨D(), Hess(φ) ⟩
       -2 ϵ^2/φ^2⟨∇_φ,D() ∇φ⟩ = 0,

which gives (<ref>).

Next, let ω() := -_τ^-1() _τ(). We then have

    ∇_· (φ̂ω()) = ⟨∇_φ̂,ω()  ⟩ + φ̂∇_·ω() 
       =  exp( -ψ / 2 ϵ) ( ⟨∇ , ω() ⟩ + ∇_·( ω() )
       - /2ϵ⟨∇_ψ,  ω() ⟩ ),

and 

    ϵ1^⊤ (D⊙Hess(φ̂)) 1 =  exp(-ψ/2 ϵ)(ϵ1^⊤ (D⊙Hess()) 1
       -  ∂ (D_ij()  )/∂ x_j∂ψ/∂ z_j -1/2⟨D(), Hess(ψ) ⟩
       +/4ϵ⟨∇_ψ , D() ∇_ψ⟩).

In (<ref>), taking the partial derivative of φ̂ w.r.t. t, and using (<ref>)-(<ref>) together with (<ref>)-(<ref>), we get 

    ∂φ̂/∂ t =  exp( -ψ / 2 ϵ)(∂/∂ t- /2ϵ∂ψ/∂ t) 
        =  exp( -ψ / 2 ϵ)( -∇_(())  -∇_(D∇ψ) 
       + ϵ1( D() ⊙Hess()) 1 + /2ϵ⟨∇_ψ, () ⟩
       +/4ϵ⟨∇_ψ, D() ∇_ψ⟩  +/2⟨D() , Hess(ψ) ⟩)
       = -∇_· (φ̂()) + ϵ1^⊤ (D⊙Hess(φ̂)) 1,

which is indeed (<ref>). The boundary conditions (<ref>) follow directly from (<ref>)-(<ref>).


Theorem <ref> in principle allows solving problem (<ref>) in the following manner. Let (φ_1,φ̂_0) := (φ(,t=1),φ̂(,t=0)) denote the terminal-initial condition pair for the system (<ref>)-(<ref>). By making an arbitrary guess for the pair (φ_1,φ̂_0), one can perform a fixed point recursion on the Schrödinger system (<ref>)-(<ref>), and the converged pair (φ_1,φ̂_0) can then be used to compute the transient pair (φ(,t),φ̂(,t)). Then, by (<ref>), we recover (,ψ), and thus (,) from (<ref>). Notice that this procedure with small ϵ>0 will yield the pair (,) solving problem (<ref>). Finally, the mapping (<ref>) in Remark <ref> recovers the solution (,) for problem (<ref>). This algorithmic framework and its convergence will be the topic of our future research.


 

















































































 
 



§ CONCLUSIONS

We considered the minimum energy joint state PDF steering problem over finite time horizon subject to the multi-input state feedback linearizable dynamics. We showed that the density steering problem can be made amenable in the feedback linearized coordinates. We derived the state feedback controller in terms of the solutions of a pair of coupled HJB and Fokker-Planck-Kolmogorov PDEs. Furthermore, we reduced this system of coupled nonlinear PDEs to a system of boundary-coupled linear PDEs. Our results are expected to lay the foundation for developing computational algorithms solving the density steering problem.







IEEEtran


